Modeling Business Information Using XBRL: the Evolution of “Patterns”
By Charles Hoffman, CPA (UBmatrix, Charles.Hoffman@UBmatrix.com)
There has been a lot of learning about how to make XBRL work over the past 10 years. There is not necessarily general consensus as to how to best build a taxonomy. However, there are things which do work, which do not work well, and side effects which may or may not be realized by those constructing taxonomies and instance documents. All this will expose itself over the coming years as more and more XBRL is put into use.
This document takes the reader through the history relating to figuring out how to build taxonomies. The purpose is to make information available so that others can think about this issue. The second most interesting set of information, in my view, is the descriptive information in the documentation which explains observations about the patterns from both a business and technical perspective. While that is very useful information, in my view the most useful information is the details relating to changes between the three different sets of patterns. Why did the patterns change? All this information is there for you. If you want to understand how XBRL works in specific situations, you can learn this from this information.
While I did physically create these XBRL taxonomies, instances, and the related documentation; I could have not done so without the gracious help of a number of people over the years including: Walter Hamscher, Geoff Shuetrim, David vun Kannon, Rene van Egmond, Thomas Egan, Josef Macdonald, Jim Richards, Roger Debreceny, Jeff Naumann, David Prather, Alan Teixeira, Hugh Wallis, Allyson Ugarte, Colm O hAonghusa, Giancarlo Pellizzari, Yossef Newman, Rob Blake, Mark Creemers, Marc van Hilvoorde, Herman Fischer, Ignacio Hernandez-Ros, Cliff Binstock, David Scott Stokes, Masatomo Goto, Paul Warren, Mark Goodhand, Campbell Pryde. There are others which I probably left off, and for this I apologize. Each of them directly and/or indirectly contributed pieces to this puzzle. I acknowledge and appreciate the thinking others contributed to this endeavor.
The first taxonomy ever created using XBRL 1.0 was released July 31, 2000. That taxonomy, for financial reporting under US GAAP, was fairly useless; however, it started a chain of events which helped to improve XBRL, to better learn how to build taxonomies, and to understand how to use XBRL to achieve specific results.
I helped to create that first taxonomy and many subsequent taxonomies. For those of us trying to build that first taxonomy the question was how do you build a taxonomy? We literally had no examples, no path, nothing to provide us with guidance.
To make a long story shorter, when we were building this first taxonomy we noticed what I started calling “patterns”. These patterns were simply the same characteristics repeated in different areas of the taxonomy. Not a grand idea, patterns exist everywhere. Patterns were things like:
· Hierarchies of concepts somewhat like sections and subsections of a book
· Calculations, things that added up
· Movements which were things that reconciled a beginning balance to an ending balance, articulating changes.
Patterns were things that occurred in business information, not an XBRL artifact. Once these patterns were identified, the question became what was the best way to turn the business information into an XBRL taxonomy and ultimately an XBRL instance document which expressed, correctly, the information one was trying to represent.
Several of us creating the taxonomy talked to different technical people trying to get their views on how to best express this information. One of the first things we noticed is that we got different answers, sometimes vastly different answers, from the different people talked with when we showed them exactly the same business situation. How could that be, we thought?
The second thing we noticed is that we could not express many of the things which we needed to represent.
I started “collecting” these patterns which we discovered. I stated to write the information down in a document. (I actually have a copy of these documents going back to January 2002 which was version 11). I was literally traveling around the world talking to different business and technical people, trying to figure out how to use XBRL correctly. I kept getting different answers. Some of the answers made no sense at all and they were disregarded. But, other answers did make sense, sometimes multiple answers seemed to make sense for exactly the same business situation. Also, there was that issue that we simply could not get some things which we really needed to work.
All this was so frustrating that I called for, and got, a meeting with a number of people providing input in how to build a taxonomy; I wanted to get as many of these people as possible into one room and have a conversation about how to best build a taxonomy.
This meeting yielded three specific things:
1. It was clear that some fundamental business use cases could not be met.
2. There were specific areas of agreement and disagreement between those providing ideas on how to model information within a taxonomy.
3. A document and a set of files which captured this information. This information could now be more broadly distributed.
My efforts continued on for a number of months, I kept tuning this information more and more. Then something occurred which made all these things more important: we started building another taxonomy, Financial Reporting using International Financial Reporting Standards (IFRS).
Hours and hours of discussions with business people trying to figure out what information we needed to express and how to express that information yielded a very honed understanding of what XBRL could express and what it could not.
During the early stages of the creation of the IFRS taxonomy, we had moved from XBRL 1.0 to XBRL 2.0. The patterns were reworked to reflect this new version of XBRL, some of the issues went away because XBRL 2.0 corrected issues encountered with XBRL 1.0. However, now new issues were being encountered for two reasons. First, the switch to making use of linkbases made working with things seemly more complicated from some perspectives as it provided additional options. Second, we started discovering new issues.
Creation of the IFRS taxonomy began in about 2001. There was much haggling as to which approach to use to organize the taxonomy. In 2002, several of us decided to take on figuring all this out and we basically took charge of the IFRS taxonomy creation project which had stalled and become fragmented. A group called “SANZ” had been informally put together (acronym for Singapore, Australia, New Zealand) because they wanted one IFRS taxonomy to exist and it was needed sooner rather than later.
We had meeting after meeting trying to figure out what business information to express and how to represent that information within an XBRL taxonomy. We keep honing and honing our understanding of taxonomies and the resulting instances which resulted given a taxonomy created to represent that specific business information.
Our understanding grew and grew, what had become known as “The Patterns Document” kept getting better and better, and the issues the business people had with expressing the information they needed to represent started becoming clearer and clearer.
These issues were being discussed more and more as people could read the document, look at examples, and see how the examples worked or did not work to meet our business needs. With discussions via email list and at XBRL International conferences, these issues were becoming clearer and clearer. Many issues were being resolved. New issues were popping up. It was clear that XBRL 2.1 needed to exist to resolve many of these issues.
Before an XBRL International conference which was to be held in Tokyo, I was asked by someone to put together a summary of the issues relating to XBRL 2.0 which needed to be dealt with in XBRL 2.1. (This was about November 2002). They asked me if I would present this information at a meeting during the Tokyo XBRL International conference. I put together and presented the information, all based on tangible examples which were quite easy to see and understand from The Patterns Document.
After the meeting, I was asked to break the one document into individual issues and requirements, which I did. I had a total of 17 issues/requirements documents (I still have these documents on my hard drive). Several meetings were held, two or three of the issues were thrown out being deemed out of scope for XBRL 2.1, several more issues were identified by others, and a total of 21 specific issues were provided to the XBRL Specification Working Group as inputs into what was wrong with XBRL 2.0 which needed to be explicitly addressed by XBRL 2.1.
Something else was going on during this time. The IFRS taxonomy was progressing and following The Patterns Document. An effort to create a revised (revise the first 2000 version) US GAAP Taxonomy had started. The US GAAP Taxonomy project did not particularly care for what the IFRS taxonomy creation people were doing for whatever reason, and people became concerned that the IFRS and the US GAAP taxonomies would use totally different approaches to creating their taxonomies.
As a result, an effort to not allow this to occur resulted in the creation of the Financial Reporting Taxonomy Architected (FRTA) and the Financial Reporting Instance Standards (FRIS). Other taxonomy efforts were in existence, but the information flow from those other taxonomy creation efforts into XBRL International was not that great. But everything about both the IFRS and US GAAP taxonomy creation projects was quite transparent, discussed on email lists, on conference calls, and at XBRL International conferences and other meetings.
XBRL 2.1 was released as a recommendation on December 31, 2003. In addition to the specification, XBRL 2.1 had a conformance suite, which is basically a bunch of tests to show how things were supposed to work and used to test software interoperability. I became quite interested in the conformance suite because I realized that that was a way to see how things worked and if things worked correctly. I used all the patterns I had collected, documented in The Patterns Document, and painstakingly created typically by hand because I was generally ahead of software’s ability to support the features of XBRL 2.1. This was a lot of work, but it helped me understand XBRL inside and out and it helped me see how to best express information using XBRL and the positive and negative characters which resulted from expressing information in different ways.
After XBRL 2.1 was released, we went back to work on the IFRS taxonomy, adjusting everything to work with that version of XBRL. One of the things which resulted from all the work with the patterns, using all the “tinkering” with the patterns to test, identify and help to resolve issues relating to the XBRL 2.1 specification was that I was named as a contributor to the XBRL 2.1 specification (see http://www.xbrl.org/Specification/XBRL-RECOMMENDATION-2003-12-31+Corrected-Errata-2008-07-02.htm.)
All the patterns documentation and supporting files had been updated, a few new patterns we discovered were added, and again those of use creating the IFRS taxonomy were able to hone our understanding of both the business information we were expressing and our understanding of how XBRL worked if information was expressed in certain ways, favorable and unfavorable characteristics. The efforts to create FRTA and FRIS resulted in increasing agreements in a number of areas; The Patterns Document and files were updated to reflect these agreements.
We continued to work on the IFRS taxonomy. I realized how much you could learn from what others were doing and the issues they were encountering in building taxonomies and working on these taxonomy projects. Another taxonomy project which was underway was the CRAS project for credit reporting. I stayed on top of that project, comparing notes with a number of people working on that project, learning about some of the unique things they were doing. One of the interesting/clever things with CRAS was that there was a lot of use of XML Schema to create solutions to meet the needs of business users. This provided me with two things: a much better understanding of XML Schema and an understanding of how many different ways there are for doing exactly the same thing. Basically, I began to see the positive and negative implications of XBRL using XML Schema to represent information.
Another project was starting, COREP and FINREP (http://www.corep.info and http://www.finrep.info). During some trips to Spain relating to the CRAS project, I met many of the people starting up the COREP and FINREP projects. Also, the COREP/FINREP folks were quite interested in the IFRS taxonomy as the FINREP project related to financial reporting by financial institutions. A big week-long meeting relating to expanding the IFRS taxonomy for use by financial institutions was held in Spain, which I attended. All this resulted in an invitation in January 2004 to participate in the meeting to establish the taxonomy architectures for the COREP and FINREP taxonomies.
A big need for COREP was one of the things which was on the list of 17 issues which I had articulated but was deemed out of scope for XBRL 2.1 which I called an inability to express relations between entity information. However, the issue was really a need for XBRL to express dimensional information in general. XBRL International had come up with a clever way to articulate this dimensional information which was XBRL 2.1 compliant. That did not quite meet COREP’s needs. Leaving out some details and making a longer story short, during the architecture meeting in January 2004 the COREP/FINREP taxonomy creation team decided to not go down a proprietary path in modeling dimensional information, but rather to work with XBRL International and create a standard solution.
This resulted in an effort to create XBRL Dimensions.
Now, new techniques for modeling information existed, and these were added to The Patterns Document. No new patterns really resulted, only new ways to model the existing business use cases.
Taxonomy work went on, work on FRTA and FRIS progressed, this forced the IFRS and US GAAP taxonomies to be more similar than different. In May of 2005, the IFRS taxonomy was released as a recommendation after years and years of effort. It complied with FRTA, it did not make use of XBRL Dimensions as they were deemed not yet really ready for prime time, and the first complete and stable version of The Patterns Document and relating files were finalized. This set of files shows how information was modeled in the IFRS taxonomy for that version of the taxonomy. This is that information:
Patterns Document (2005)
· Patterns Documentation: http://frux.wikispaces.com/space/showimage/Chapter-11-ModelingFinancialReportingConcepts.pdf
· File Set for Download: http://xbrl.squarespace.com/storage/Patterns-2005-07-07.zip
· Human browsing: http://www.xbrlsite.com/patterns/2005-07-07/Viewer.htm
· RSS feed: http://www.xbrlsite.com/Patterns/2005-07-07/rss.xml
(Note that this information was eventually published within the FRUX book, the best version of patterns documentation is Chapter 11 from that book, see http://xbrl.squarespace.com/financial-reporting-using-xbrl/).
The first truly useable version of the IFRS taxonomy was released, the US GAAP project was still under way, FRTA was released as a recommendation, FRIS was released as a public working draft.
If one reads The Patterns Document, one can see issues which still remained unresolved at this point in time. The issues are documented within the explanations of each pattern. Also, there were still a number of different options of expressing information, limitations and other issues relating to using tuples were becoming apparent, and XBRL Dimensions was starting to cause additional lack of clarity as to when to use tuples and when to use XBRL Dimensions to model business information.
The second version US GAAP Taxonomy was completed (2005). Trying to get that taxonomy to achieve FRTA compliance resulted in a number of areas of understanding. First, it showed that software was still not interoperable enough. Second, it showed that you could have different taxonomies be FRTA compliant but still significantly different and not necessarily interoperable.
After the second version of the US GAAP Taxonomy was released in 2005, everyone knew there would be a third version of the US GAAP Taxonomy and work on how to get it created started almost immediately. The US Securities and Exchange Commission was becoming interested in using XBRL, but the taxonomy created by the US’s volunteer effort was moving too slowly and the taxonomy was just not covering enough of US GAAP. This was because the taxonomy had no substantial funding behind it, the work was being done by volunteers who worked on the project when their day jobs allowed.
The SEC’s involvement resulted in one thing all these taxonomy creation project lacked: funding. A new project was to be created, the SEC would fund it, and another US GAAP Taxonomy would be created. Prior to officially starting that project, everyone knew that a lot of disagreements as to how to create that taxonomy need to be resolved. Two projects were undertaken by XBRL US to help resolve these issues. The first was to create a US version of the patterns document so people could agree on how to model the information within the new US GAAP taxonomy which was to be constructed. The result of that project was a lot of discussions, increased insight, additional agreement on certain issues, and the following set of documentation and supporting files:
USFRTF Patterns (2007)
· Patterns Documentation: http://www.xbrl.org/us/USFRTF/USFRTF-PatternsGuide-PWD-2007-04-17.doc
· File Set for Download: http://www.xbrl.org/us/USFRTF/USFRTF-PatternsFiles-PWD-2007-04-17.zip
· Human browsing: http://www.xbrlsite.com/patterns/usfrtf/viewer.htm
· RSS Feed: http://www.xbrlsite.com/patterns/usfrtf/rss.xml
These patterns basically started with the IFRS patterns, modified those patterns to meet the desires of and agreements reached by those creating this document, and it added a few additional patterns and modeled some business information using XBRL Dimensions.
Subsequent to the release of this document but included in the set of files was the addition of a modeling of every tuple within the set of files using XBRL Dimensions to see what that looked like. The purpose of this was to understand the differences between the characteristics of tuples and XBRL Dimensions.
The second project which was undertaken was to come up with what was called a “style guide”. The style guide basically documented agreement on how to label concepts, creating additional consistency in that area of a taxonomy. You can see a copy of that document here:
· US GAAP Taxonomy Style Guide: http://www.xbrl.org/us/usfrtf/XBRL-StyleGuide-RECOMMENDATION-2007-03-08.doc
The way XBRL US operated was changed beginning in 2007. It had operated as a group of unpaid volunteers. XBRL US changed that and began to fund projects.
There were still issues which were not agreed to which would impact the US GAAP Taxonomy. Everyone knew that if the disagreements continued, all it would do is delay the creation of the US GAAP taxonomy. In an effort to get all these disagreements out on the table and resolved a meeting which was referred to as “The Cage Match” was put together. Basically, everyone who cared about these issues was given an opportunity to sit in one room in New York City and resolve the issues. There were about 20 people who were to participate in this meeting.
Having been to a number of these types of meetings before, I understood that there was a high probability that the meeting could end up being a waste of time. To avoid this possibility, I created a document which I called “The Choices Document”. That document was a collection of all the known issues which were outstanding, all the know options which could have been picked, and a proposed resolution with reasoning to support that proposed option. The document was circulated prior to the meeting so those coming to the meeting could be familiar with the issues and potential options. (If you want a copy of this document, please email me and I can send it to you, the information is quite good.)
The Cage Match was held over a three day period. Virtually all the issues were resolved unanimously which was unprecedented. Getting a US GAAP Taxonomy became just a little easier.
Around this same time a group called “TAG” or the Taxonomy Architecture Group was established. The purpose of the group was to come up with an architecture for the US GAAP Taxonomy. It was constrained by the results of The Cage Match. It was to propose that architecture to XBRL US’s Domain Steering Committee. The group selected for the TAG was: Cliff Binstock, Walter Hamscher, David vun Kannon, and myself. Campbell Pryde and Paul Sappington also participated as they were on the XBRL US Domain Steering Committee.
One of the group’s outputs was the US GAAP Taxonomy Architecture document (see http://xbrl.us/Documents/SECOFM-USGAAPT-Architecture-20080428.pdf). This version is quite interesting and has a lot of information about what that group considered a good architecture. (Note that there was a public working draft released for comments on July 7, 2007 which has additional information which was dropped from the final version. If you want a copy of that, please email me and I can send it to you, this is public information. It contains a lot of very good details dropped from the final version of the architecture document.)
I am not going to go over the entire architecture, but I do want to mention two things which are pertinent to the next phase of taxonomy patterns. First, the notion of “disciplined extensions” and “compact patterns declarations” were discussed in the final and proposed architectures. It is best to read the architecture document to completely understand these two notions. These are definitions of these two concepts from the architecture documents of the US GAAP Taxonomy:
Disciplined Extensions – Version 1.0 internally enforces design rules to ensure that the base taxonomy from which others will need to extend is internally consistent. It is beyond the scope of version 1.0 to create a formal expression of extension rules to facilitate "disciplined" or "channeled" or "managed" extensions within systems that use it. We encourage systems that make use of version 1.0 to build such formal expressions for use within their systems. The Compact Patterns Declarations (CPD) is an example of such formalized expressions for the purpose of managing extension by filers.
Compact Pattern Declarations – The UGT will enforce, as robustly as practical, protective measures to build both (a) a consistent base taxonomy from which to extend and (b) mechanisms to control extensions or to point users extending the UGT in the right direction [CPD]. The formal expressions of this idea are the compact pattern declarations used to consistently create the base taxonomy. In addition, extensibility rules built into these CPDs can be used to assist in the integrity of the extensions. CPDs also provide extension points themselves, acknowledging that retaining flexibility over the long term is important given the current level of maturity of XBRL technology and methodology.
The second noteworthy event worth mentioning is that these two notions were suggested by the taxonomy architects, but never completely implemented in the taxonomy for various reasons which I cannot get into.
But these were critical concepts to understand and they literally had significant impact on whether a system would work, or not work, as needed. As such, myself and several others which saw the value of these “compact pattern declarations” and the notion of “disciplined extensions” continued work in that area. We created new versions of patterns documentation and files, leveraging the USFRTF Patterns. We also changed our perspectives slightly, creating meta-patterns and changing the patterns more from a learning tool to a more formal documentation of the information model of taxonomies.
Rene van Egmond and was also noticing some similar issues relating to other taxonomies and instance documents. Rene was working on a paper to compare taxonomies. He had worked on the Netherlands taxonomy project and was trying to help the Australian Standardized Business Reporting Project get going. The Australian project was leaning toward following the Netherlands project, not really considering all the learning which had taken place on the US GAAP Taxonomy project.
Rene gave me a copy of his paper and I started editing it with the observations I had made from my participation on the US GAAP Taxonomy project. Over the next several weeks, the paper evolved into the white paper “XBRLS: How simpler can be better”. You can get a copy of that whitepaper here: http://xbrl.squarespace.com/storage/xbrls/XBRLS-How-simpler-can-be-better-2008-03-11.pdf.
This resulted in another iteration of the “patterns” (more information models) called XBRLS (see http://xbrl.squarespace.com/xbrls/). These patterns, meta patterns and other information were created at the same time the white paper was created. XBRLS, which is a bit of an unfortunate name because people misunderstood it and read many different things into the name, is heavily influenced by the US GAAP Taxonomy Architecture and all that had been learned about taxonomies to this point in time.
The US GAAP Taxonomy Architecture was heavily influenced by what had been learned from the IFRS taxonomy, COREP, FINREP, older versions of the US GAAP Taxonomy, the Netherlands project, CRAS, and several other taxonomy projects. All the pros, cons, good, bad, what worked, what did not, was all condensed into one approach to how to build taxonomies.
XBRLS Business Use Cases (2008 http://xbrl.squarespace.com/xbrls)
· Patterns Documentation: http://xbrl.squarespace.com/storage/xbrls/XBRLS-BusinessUseCases-2008-04-25.pdf
· File Set for Download: http://xbrl.squarespace.com/storage/xbrls/XBRLS-BusinessUseCases-2008-04-18.zip
· Human browsing: http://www.xbrlsite.com/Patterns/2008-04-18/Viewer.htm
· RSS Feed: http://www.xbrlsite.com/Patterns/2008-04-18/rss.xml
The term “pattern” was dropped in favor of information model or information modeling patterns.
There was something missing with all these patterns, two things really that I could see. There may be more, but two are obviously missing.
First, if you looked at each of these patterns individually, they really did not tell you the full story. So, what I did was to combine all the patterns into one taxonomy and one instance document. I called that “the comprehensive example”. The comprehensive example helps one see how the individual pieces of a taxonomy worked (or did not work) together. I made this as simple enough to be easy to work through, but comprehensive enough to look like something real. What I did was basically create a financial reporting type taxonomy and an instance document which looks like a financial statement, containing every business use case of XBRLS. You can find that example here:
http://www.xbrlsite.com/examples/comprehensiveexample/2008-04-18
Finally, having just one instance document left one other thing missing also. What about comparability across instance documents? As such, I have created multiple instance documents and placed them into a “repository” or collection of XBRL instance documents. See more about repositories here: http://xbrl.squarespace.com/journal/2008/11/13/thinking-about-repositories-of-xbrl-information.html
These repositories were modeled somewhat based on the SEC’s RSS feed to XBRL instance documents: http://www.sec.gov/Archives/edgar/xbrlrss.xml
Here are two comparable repositories (the information is meant to be compared):
http://xbrl.squarespace.com/storage/samplerepositorymini/rss-RepositoryMini.xml
http://xbrl.squarespace.com/storage/samplerepositorybasic/rss.xml
These are repositories of un-comparable instance documents. They are basically the patterns instance documents.
http://www.xbrlsite.com/Patterns/2005-07-07/rss.xml
http://www.xbrlsite.com/patterns/usfrtf/rss.xml
http://www.xbrlsite.com/Patterns/2008-04-18/rss.xml
Here is an RSS feed which chains three repositories of XBRL instance document together so software can work with them as a set.
All RSS Feeds: http://www.xbrlsite.com/rss.xml
Well, that is where the journey takes us to today. There is definitely still more to come!