Do you know your data?

Some things never cease to amaze me. BusinessIntellienge (according to Wikipedia) has been around for over 50 years. Despite this fact, all too often companies are inadequately providing the basic foundation for Business Intelligence. Here is one small example….knowing what data you use to make decisions. Whether it is a government organization, public company, or private company, they typically know what information they need to make their most important decisions, at least from a business perspective, however, from a data perspective, IT typically has very little insight into what data is actually being used.

Let’s check out a typical example. I have seen countless times where companies are implementing new systems that require major database changes. Well, when these changes are made, IT typically has a very difficult time determining exactly what data is being used, by what reports, by which users, not to mention how many times. Now if you think about it, this is very important information. In our society now, it is easy to get lost in data, there are so many sources of information, such as this blog, news sites, facebook, myspace, linkedin, drudgereport. I think we all can agree that the problem is not if the information exist, it is, can we find it and if we can, can we connect to it. So why are the questions about the data so important?

What data is being used? By whom? How often? This is important, especially when implementing solutions such as a data warehouse. Data Warehouse implementation can be very costly and often fail even after spending hundreds of thousands of dollars above what the budget required. Many of these failures can be prevented by the ability of knowing the answer to these questions. I have been working on a project where I an organization has to transfer 200 reports that are on their legacy system and have recreate them using Crystal Reports on their new ERP system. The process is to analyze each report, find out what fields were used to create that particular report and then look to see if the data was converted to the new ERP system. What sense does this make? All of these reports are considered mission critical by this organization, so my question is why is a field in a report that is mission critical no longer available in the new ERP system. A best practice approach could have been taken here before the data conversion process if the organization new which reports were mission critical and were able to generate a report that listed all the fields within those reports.

While I was at Information Builders, Inc. using WebFOCUS, they had a fantastic product called Resource Analyzer that at a click of a button, you would have access to hundred of reports that gives you the ability to analyze what data is being used, by whom, how often. Not only that, but it could also provide you with some what if analysis…for example…what if I removed this field from our database? What reports would that affect? What users would be affected? I don’t know about you but before I make any major changes I definitely want to make sure I know the impact it will have on upper management.