Garbage in, garbage out – Making sense of your data
March 16, 2017 by Jacob Stoller
One of the most important IT trends happening in construction right now is that of data analytics. The practice of analyzing stored company data to help make business decisions, is gaining ground in companies of all sizes. Even a contractor with several pieces of earth moving equipment, for example, can collect data from the equipment’s recording devices and use it to determine which operator or crew is most productive.
The challenge is that collected data is not necessarily usable data. If the data is of poor quality, that is, it contains errors, redundancies, or variations in how it is stored; this can lead to users receiving incomplete or wrong information.
Such discrepancies can be difficult to detect – data is an intangible and invisible entity that can only be properly visualized by highly specialized IT professionals. The unseen, however, can be dangerous – data quality issues account for some of the thorniest IT problems.
No contractor should try to become an expert in this highly technical area, but everybody should be aware of the high level issues. Here, based on the advice of experts, are a few best practices for managing some of the more mysterious aspects of data.
Store data in unified formats
Today, it’s easy to store data in a variety of formats. It is difficult, however, for software applications to extract information from data when formats are inconsistent. “Businesses may be storing data in a wide variety of formats such as pdf, web pages, word, excel, etc.,” says Chris Kimberley, manager, data architecture and problem management at EllisDon, “It can be difficult to derive insight from these different formats. A strong data governance presence can ensure that businesses are collecting their data in unified formats that make life much easier when it’s time to perform analytics.”
Don’t store what you don’t need
The ease of collecting data has led many companies to store far more than they can handle. “You don’t want to be storing data that you’re not going to use in the future,” says Andrew Lockwood, data architect for Toronto-based construction software provider Bridgit. “Extra data is noise. It costs you money, and it makes it harder to get accurate results.”
Match the type of database to your business
Not all databases are created equal. A big trend to be aware of is the rise of NoSQL, which is an alternative to the traditional relational databases such as the familiar Microsoft SQL Server, which have been the industry standard for decades. The difference from a business owner perspective is that NoSQL is more flexible in terms of inputting and storing diverse forms of data, while relational databases are more rigid in that respect, but have much more flexible and powerful reporting capabilities.
A smaller contractor with a portfolio of diverse applications and minimal reporting requirements might find NoSQL a better choice; while a larger company operating an integrated software suite might be better off with a relational database.
Integrate applications wherever possible
Diverse applications tend to store data in different formats, and this makes it difficult for companies to share common data such as employee information and financial numbers. This necessitates a lot of manual re-entry work, and increases the chances for errors. It is also one of the strongest arguments for purchasing an integrated construction software suite, or working with a vendor that can integrate various components to provide a reasonable degree of integrated functionality.
Maintain control of your data
Due to substantial investments in security, major cloud vendors have convinced many doubters that the cloud is now safer than on-premises data storage. The issue of control, however, remains. “It’s easier for database administrators to have data stored in house as you have the data in its raw format, and full control,” says Kimberley. “However, the potential benefits of utilizing Cloud-based services can outweigh the benefits of simplicity in-house.”
As well, some vendors extract information from data under their care and create benchmark reports and other business intelligence. “You can be signing away the rights to your data if you’re not careful,” says Jagodich, VP and CIO at EllisDon and president of construction software provider Gate Three. “Are they going to aggregate it with everybody’s data, take some intellectual information there and sell it back to people?”
Database problems, unlike system crashes or network issues, can be difficult to detect, and when they persist, they get harder to fix as more bad data accumulates. Decision makers, Lockwood suggests, should make a habit of looking carefully at business intelligence reports to confirm that they make sense. “You might poll your data, run your queries, put the information into a graph, and then see that it doesn’t match reality,” he says. “So if you add some reporting functionality, use it right away, and then keep an eye on it and monitor it. Is it always doing what you want? If not, immediately dig into why. If you let issues go for some time, it can be so much harder to correct after the fact.”
The bottom line
Probably the most important thing to remember is that data, depending on how it is used and managed, can become very valuable, or a serious liability.
“It’s important to recognize that data is an asset, and you have to take the same type of attitude towards data as you would any other asset of the company,” says Jagodich.
Jacob Stoller is principal of Toronto-based consultancy StollerStrategies. Send comments to firstname.lastname@example.org.