Like a good meal cannot be cooked without a good food no matter how good the cook is, an organization cannot function without good data even if with the best managers.
Data quality is a decisive factor in decision making and the overall functioning of the organization. It’s not the only one, but a very important factor. In practice, there is a proven relationship of data quality with the successful operation of the enterprise - poor quality data costs companies around the world each year hundreds of billions of dollars.
Hand on heart. How many times have you looked something for, repaired, or you did something wrong, because you had bad information or you did not have any information at all. These are all consequences of poor quality data.
What is data quality and how to measure it?
View of what actually data quality is and how to measure it is constantly evolving. Because this is not an exact discipline, there are several approaches on how to define data quality and how to measure it. Data quality is a wide range of dimensions - you cannot simply say that quality data is the error free or the most accurate or the most up to date. In each view there is something needed, and for each data we have another list of characteristics by which we can judge whether it is a quality data or not. Data quality therefore cannot be assessed by only one indicator. Very often cited list of data quality dimensions is provided by COBIT, which defines the following dimensions of data quality:
But there are many other lists of data quality. Perhaps the most comprehensive list of dimensions of data quality was the result of the MITIQ from MIT.
Which enterprise data must be of high quality?
We could say all, but the greatest emphasis of course is placed on business and financial processes which directly affect the existence and health of the enterprise. This is the data for key processes such as sales, billing, delivery of services and products to customers, production or profitability. Sending the wrong amount on the invoice may be equally fatal for the organization as well as sending the correct amount to the wrong customer or to the wrong address. Data quality relates to transaction data (invoices, orders, requirements, etc.) and all master data, such as data about customers, suppliers, business assets or employees.
What are the consequences and impacts of poor data?
A poor data and information quality causes problems almost everywhere you look. The most obvious problems are in repeated processes and activities, which generate additional “muda” workload due to poor information inputs, inconsistent, poor or even conflicting data or reports from various source systems and applications (such as accounting systems , ERP systems , CRM systems ). In a research carried out by MIT in 2006, it was stated that even 77 % of lost deliveries were due to poor data quality.
Common impacts of poor data:
- Repetitive processes to no purpose, extra work caused by poor quality information inputs, tracing and inefficiency
- Errors in reports due to duplicate or inconsistent data
- Returned mail, delivery or needed communication due to incorrect data about customers
- Duplicate or overlapping master data about customers or products
- Loss of knowledge of the company due to leaving of employees and staff
- Loss of customer confidence due to his negative experiences
- Missed deadlines for the conclusion of the financial statements and other corporate reports
- Increased risks due to incorrect decisions
What are the causes of poor quality data?
Some companies hope to improve data quality by going to large systems such as ERP or CRM. Other organizations use tools for data cleaning in data warehouses to find dirty data and could subsequently cleaned it using ETL tools. All of these technology-based methods to improve data quality are laudable and certainly a step in the right direction. The mere technological solutions however cannot annihilate the roots of poor quality data because it is not only IT problem, but it is primarily a problem of the whole enterprise architecture, especially well or poorly designed processes.
Common causes of poor data:
- Poorly adjusted and managed processes
- Poor rules and management
- Poor discipline of people (in data storage and management processes)
- Improperly deployed enterprise applications
- Complicated enterprise applications
- Poor or inadequate functionality of enterprise applications
How to solve data quality in the organization. Remove the causes, not just the consequences!
All business processes and activities must develop, implement, and require quality data and information. All together. Data quality must be addressed, as well as other quality as a property of the whole system, as part of all processes. It’s not a one-off event, although they are necessary to start the whole long-term process of quality improvement.
Data quality must include a number of cross-cutting activities that should be part of quality improvement throughout the process. It is necessary to set the overall control system so as to prevent negative effects and poor data quality. It is necessary to use principles of quality management and similar approaches, such as PDCA , Kaizen and more.
- A strong commitment and support of data quality at the highest management level
- Quality-oriented incentive system
- Directives requiring data quality
- Audits of data quality
- Increased training for data owners about their responsibilities
- Maintenance of metadata and data management techniques
A piece of advice at the end. It is always better to “manufacture” quality data at the beginning than check its quality and clean dirty data afterwards.