Data Quality, Data Quality and Data Quality
During the last couple of months I had the pleasure to speak with many data management experts. Actually the passion for data is overwhelming and the conversations lively. Many topics come to the table and we all do things different but with a similar purpose. This purpose is contribute to the success of the company you work for by managing data in the most optimal way. One thing we have all in common and it keeps on coming back and that is data quality. The data quality challenge is the hardest one for all of us.
What is data quality?
Before you begin to get a handle on the data itself, it’s important to understand what “it” is. According to another Gartner study, data quality is examined by several different points, including:
- Existence (does the organization have the data to begin with?)
- Validity (are the values acceptable?)
- Consistency (when the same piece of data is stored in different locations, do they have the same values?)
- Integrity (how accurate the relationships between data elements and data sets are)
- Accuracy (whether the data accurately describes the properties of the object it is meant to model)
- Relevance (whether or not the data is appropriate to support the objective
Bad data quality costs your company money
Different researches concluded that bad data is a killer among us. It affects complete company performances.
Harvard Business Review wrote that bad costs us $3.1 trillion, IBM’s estimate of the yearly cost of poor quality data, in the US alone, in 2016. While most people who deal in data every day know that bad data is costly, this figure stuns.
For insurance the estimated the cost (of bad data) is between 15-20% of operating revenues. Insurance Data Management Association,
Bad data cost around 20% revenues/operating budget. Australian IT news, 2011
These are just some examples to take in mind how worse the situation really is. The consequences are enormous that is something we all agree on.
As described in the Book: Data Quality: The Field Guide by Thomas Redman PhD the subject is not really exciting. However just take a brief look at numbers above and you will change your mind. He describes is as follows in his book introduction:
Can any subject inspire less excitement than “data quality”? Yet a moment’s thought reveals the ever-growing importance of quality data. From restated corporate earnings, to incorrect prices on the web, to the bombing of the Chinese Embassy, the media reports the impact of poor data quality on a daily basis. Every business operation creates or consumes huge quantities of data.
Also the blog from Henrik Liliendahl on the Data Management Experts Platform focuses on this topic from a different angle: Data Sharing Is The Answer To A Single Version Of The Truth. It is a topic that is on our minds during the day. Many software vendors have made it their business to support your company in this area of expertise. However the discussion is still alive. Is it a business issue or an IT issue?
In my opinion Data Quality is a business issue which can only be tackled when IT supports the initiatives taken by the business. Acting as a team with the same goal is the key. Also companies have to make effort to take data quality serious. Putting it on the agenda of the board meetings is not that strange. They also want to make decisions based on reliable data. The ultimate key to success could be data governance, not just basic data governance around your master data but data governance through the whole company and even including the Big data sources in it.