Dirty Data Jargon Buster

What is Dirty Data?

Dirty data is an expensive liability which is costing UK business up to £900 billion a year. Research from Experian suggests that 22% of customer records within the average database are inaccurate with retail and education having the highest associated costs.

In our foundational article - Data Cleansing with the Medallion Method we take you through the process of ingesting and cleaning your data ready for analysing and reporting. In this article, designed to be read alongside, we give you some handy definitions and examples. Let’s start with what we mean when we talk about dirty data...

Dirty Data Definition

Dirty data is basically inaccurate, incomplete, or inconsistent information in a dataset, often caused by human error, outdated systems, or lack of standardisation. If it is not cleaned up it can lead to poor decision-making, lost revenue and damaged customer trust. 6 categories of Dirty Data you might recognise... Duplicate Data: Multiple records for the same customer, leading to several emails going to the same person and incorrect analytics.

Inaccurate Data: Misspelled names, wrong phone numbers, or incorrect bank details, causing failed transactions or communication errors.

Incorrect Data: Values that are outright wrong.

Incomplete Data: Missing fields like customer email addresses or shipping details, resulting in delivery delays or missed opportunities.

Outdated Data: Old addresses or job titles, leading to irrelevant marketing campaigns and wasted resources.

Inconsistent Data: Different formats for dates or currencies across systems, complicating reporting and decision-making.

Misleading Data: Deliberately inaccurate information (e.g. underreported income).

Siloed Data: Information isolated in departments (e.g. sales vs. customer service), preventing a unified view of customer interactions.

In our jargon-busting guide to data terms we take you through a lot of terms you will come across when working on your data storage and cleaning strategy – here are a few more pertaining specifically to the data cleansing task at hand:

• Data Cleansing: The process of identifying and correcting dirty data.

• Data Validation: Ensuring data meets specific criteria at entry.

• Data Enhancement: Adding related information to make data more complete.

• Normalisation: Standardizing data formats (e.g., expanding abbreviations)

• Fuzzy Matching: Correcting records that partially match known entities

The Impact of Dirty Data

Letting dirty data into your system can have serious consequences which are simply not worth the risk. It’s like putting petrol into a diesel car – costly and inconvenient. For example, if your business revolves around dealing with customers then accurate contact details in your CRM are going to be paramount.

A notable example occurred with police stop-and-search data. When it was imported from the UK police data portal into Excel, some age category data appeared in the wrong column and was misinterpreted as a date, leading to incorrect analysis and reporting.

Here are some of the potential repercussions…

• Financial Loss: Businesses lose millions annually due to poor data quality.

• Customer Dissatisfaction: Inaccurate data leads to irrelevant communications.

• Operational Inefficiency: Misallocation of resources and lack of innovation.

So, if you are ready to get your data ship-shape and squeaky clean take a look at our foundational article - Data Cleansing with the Medallion Method which will give you a good idea of how it’s done. Then you’ll be ready to talk to your data team or come to us for some consultation – we’d be more than happy to help.