Written by Hugh Wallis Posted on December 10, 2008
Hugh Wallis has overall technical responsibility for standards development at XBRL International. He became involved with the XBRL Consortium in January 2000 while a software architect at Hyperion Solutions. Mr. Wallis was formerly Chair of the XBRL Specification Working Group, Chair of the XBRL-GL Working Group,and Chair of the XBRL Canada Domain/Taxonomy Working Group. He is a co-editor of the XBRL 2.1 Specification and other XBRL International technical products. Mr. Wallis can be reached by email.
The process of creating technical standards is one that is typically misunderstood by almost everyone -- even many of those deeply involved in it. The development of XBRL is no exception to this phenomenon. I hope this post sheds some light on how XBRL gets developed and helps dispel some of the misconceptions about this process.
There are a number of factors at play generating these misconceptions, and probably the most influential is one’s experience with software products. When a commercial company decides to produce and sell a piece of software, they need to bring it to market as fast as possible in order to start generating revenue; they then continue to develop and improve it based upon customer experience and feedback. This means that, following the alpha and beta stages (which are generally incomplete versions that are distributed to early adopters), a first version – Version 1.0 -- gets shipped that generally has some of the planned features not implemented. More important, there will usually be a number of known bugs – known to the developer, that is, but not necessarily publicly acknowledged. Then, with some money coming in the door that can be used to fund further development, coupled with actual implementation experience from customers, new versions are created and released on a regular basis.
So the marketplace is told that the 1.0 version is a “finished product” and is generally prepared to accept that at face value. Although the company usually continues to ship versions 1.1, 1.34, 2.0, 3.0 and so on, possibly for many years to come, the act of shipping the 1.0 version was a signal to the market that the product was “ready for prime time” (even if it wasn’t – which, regrettably, is sometimes the case).
Now, let’s contrast this with the process of developing a standard such as XBRL. A critical difference between a piece of software and a standard is that it should be very difficult, if not impossible, to change a standard once it has been RECOMMENDED by the body that promulgates it. The whole idea of a RECOMMENDED standard is to establish a stable environment that all stakeholders can rely on for a long time to come and which software developers can use to go and develop their products. Everyone involved can thus have a reasonable amount of confidence that those products will be able to talk to each other.
As an example, imagine a standard for light bulb fittings had been released (ignore the regional differences among continents with respect to screw thread, voltage, and so on). With this standard in mind, companies had gone out and created lamps, bulbs, and so on, and then it was decided that it should be changed to address some issue that arose during deployment. The huge investment made by many companies and consumers would be wasted and confusion would abound. So the key difference here is that the development of a standard actually comes to an end when it is RECOMMENDED, whereas the development of a piece of software could theoretically continue indefinitely.
It is precisely to avoid this kind of confusion and waste that the standards development process is what it is, even though that may appear slow and bureaucratic to the casual observer. There are, however, a number of parallels to the software development process described earlier that I’ll discuss.
The first time a standard under development is exposed to the public is in what is called a Public Working Draft (PWD). This may be an incomplete draft that might only satisfy some of the requirements and have some features which could be changed or removed altogether. There are generally a series of PWDs as the work proceeds.This is roughly equivalent to the alpha and beta stages in software development. As the PWD process draws to its conclusion, a last call is issued for any comments preparatory to promoting the work to Candidate Recommendation (CR) status.
A CR is the equivalent to a 1.0 version of a software product. It is complete (in respect to addressing all the requirements except those specifically listed as being “at risk”) and should be capable of being implemented by software vendors. This is really the point at which the marketplace should be thinking of adopting it, just as they would probably buy and deploy version 1.0 of new, cool software. Of course, a standards body such as XBRL International is nonprofit and does not generate its revenue from selling the standard; that aspect of the similarity -- and the motivation to “ship ASAP, preferably before quarter-end”-- does not hold.
At about this time, a call for implementations is made to get actual practical experience with the CR and weed out any issues that could affect stability of the final standard. This is roughly equivalent to getting user feedback on a 1.0 version of software. Additional CRs are generally published to address such feedback (akin to versions 1.2, 1.3, 2.0 etc. of software) and when at least two interoperable implementations are available and proved, it is time to move forward to the last step. This CR process could last quite a long time because it is only by actually implementing the standard, testing it in real life situations, and making necessary adjustments that one can be confident that it can be frozen as a RECOMMENDATION. Software is usually not considered to be “mature” until it has gone through a number of releases; the same applies to a standard, with the key difference being that once you have frozen a standard, it really is frozen.
Finally, the standard achieves RECOMMENDATION status; the only changes that should happen after that are minor errata corrections. If there is much that is more serious than that, then the consortium has not done its job properly.
I hope this has shed some light on how a standard such as XBRL gets created. If you want the detailed rules about the process, feel free to ask me, or, if you are feeling brave, you could even read the process document.