An incremental approach to data conditioning works best

It's one of the truths about data that data can't take care of itself — especially when some or all of it has been collected in legacy environments and/or with little or no control over how it was collected. Taking care of your data, what's known as data conditioning, is valuable regardless of whether you're planning system modernization, data migration, or any other large, data-related project. While data problems can be huge and can seem insurmountable, you will have success with data conditioning if you address these problems in small, manageable pieces. This incremental approach, which Qbase uses and has had much success with, lets you expose critical information about the data is the most effective way. Thus, informed decisions can be made concerning risks: Do we move ahead with the data in its current state? Or do we first condition the data for improved results? We recently added another tool to assist in conditioning of your data over time. Qbase Data Drift Analysis™ allows you to take snapshots of the condition of your data at different points in time, and then to compare those profiles of your data. Identifying trends in Data Drift Analysis™ allows you to correct flawed software or procedures which may have a negative impact on your data — all this before the impact becomes a serious data quality issue. So, the bottom line when it comes to data is this: To avoid data-related risks and surprises, establish an ongoing data profiling and conditioning process to be proactively aware of the state of your data. At Qbase, we have the expertise and tools, such as Data Drift Analysis™, to lend a hand.

A bit about our standard data improvement processes

One of the primary areas of focus here at Qbase is data, specifically on what is known as "data hygiene" — literally turning dirty data into clean data. When we receive data from clients requesting data hygiene either as a standalone process or as just one step in a larger project (such as a data migration or integration project), the first thing we always do is examine the data (typically this involves using our organic software, such as Qbase Data Discovery™). We then process the data files, which almost always results in a significant improvement to the data. Some of the standard processes we perform to enrich and improve our clients' data include:

• De-duping records to remove or correct duplicate records
• Distinguishing an individual record (a person) from a business
• Parsing person names into standard fields; in other words, identifying and breaking out the title (Adm.), first name, last name, suffix (Jr.), and so on
• Identifying invalid or undeliverable addresses and correcting or updating these addresses where possible
• Parsing address components into standard fields; again, breaking out the house number, the pre-directional if used (East), street name, the suffix (NW), and all of that

If you have dirty data, Qbase has an unmatched data hygiene system to clean your data, which is the first, best step for any data-related project.