A New Look on how Business Intelligence Can Fix Your Data Quality Problems
Implementing a business intelligence environment against data you don’t trust? Sounds counter-intuitive, some would say: ‘just plain wrong.’ How could you possibly build a data warehouse and cubes on data that might not be accurate? You shouldn’t of course, and that is the point. Accurate data is the non-negotiable cornerstone of any BI effort.
But accurate data does not happen by itself. Data needs to be enthusiastically governed to a state of truth and reliability, and this is a big job. So big in fact, that attempting a global data governance initiative can cost so much time and energy as to obliterate the initiatives it was designed to support in the first place.
If your company has an urgent need to manage to 3 or 4 KPI’s around inventory, the 6 months that it might take to understand your inventory data and make sure it is correct is a very long time.
But there is an even bigger risk. Think about how you find out that you have bad data. It is almost never by putting data into a system. You only find out about bad data when you try to get data out. So this begs the question – if you are going to embark on a data governance initiative before you roll out BI, what exactly are you going to govern? How do you know what to fix, if you can’t see that it is broken?
Nothing highlights inconsistencies in your data like trying to build a dashboard against it.
Nothing will focus data governance and cleaning efforts like clearly understanding the numbers you need to drive your business. Not just the numbers themselves, but what are the components of those numbers? What were the transactions that placed those numbers into the system? What are the processes around those transactions? Are they contributing to accuracy, or is there something buried in there that is making the numbers faulty?
There is magic in seeing those imperfect numbers appear in an analysis because now you have a clue about what data is broken. BI will give you an effective data governance roadmap, with the starting point in the form of numbers that your subject matter experts and other stakeholders will recognize as wrong. It is the BI environment itself that makes your numbers accessible. And with that accessibility, data quality issues become glaring and the root causes behind them become easier to trace.
Implementing BI typically comes in the form of setting priorities: ‘we need to gain visibility and control over our AR balances first.’ ‘Next we need to tackle inventory, then sales.’ It is an iterative approach to requirements gathering and BI artifact delivery where you see the data flowing out of the system in report form, so you know what to tackle, piece by piece, in terms of data quality.
When companies install a BI solution, they should be prepared to discover that it will highlight all the inconsistencies in their data. A common one is sales orders that are posted without being attributed to either a customer or a salesperson – revenue is unaccounted for.
‘How can this be?!!” is the usual refrain. ‘We have business rules around posting invoices that are supposed to prevent this from happening.’
Whether it was an innocent work-around or a full-on policy breach, something, somewhere caused those business rules to be violated. What once was a hard-to-unravel mystery is now easy to spot, and easy to solve.
These types of discoveries are very hard to make otherwise, where the tendency is to look at data governance as a whole and try to fix the data in the same holistic way. That becomes an overwhelming project that can stop other initiatives in their tracks.