Clients Have Data Problems

No Project Has Perfect Data

For jaded and road-weary consultants, the statement that no project has perfect data sounds like an understatement. In practice, data collection resembles an Easter egg hunt where the team has a general idea where the eggs might be but cannot be certain until they start looking. The search spans multiple IT systems, hard drives, filing cabinets and dashboards. Every consultant carries a story of late nights piecing together data from improbable sources, and those stories rarely get easier over time.

The irony is that lack of data often creates the consulting engagement itself. If the client could find the data, they would have solved the problem already. Seth Godin captured this dynamic in a blog post about perfect problems, noting that the only problems left are the perfect ones, because the imperfect ones with clearly evident solutions were solved long ago if they mattered. 1 Consultants should be weirdly thankful for this dynamic, because the client's data mess equals work, projects, billing and revenue.

Most Clients Struggle With Small Data

While there is abundant talk about big data and its predictive analytics revolution, the reality is that many organizations have trouble patching together their small data. Only about one-third of companies can easily satisfy a consulting team's data request. The gap between the promise of big data and the reality of small data is where consultants spend most of their time, and bridging that gap is itself a deliverable.

Collecting data can be painfully slow. Even when the location of the data is clear, it is fairly common to spend several days hunting down the right people to extract it. Consultants often visit the client site just to request, pester and nag for the data they need. Clients could save 5 to 10 percent of fees simply by delivering data faster, yet few recognize the cost of delay.

Good Data Is Hard to Find

In experience, the larger, more geographically dispersed and older the company, the messier the data. Using the analogy of data flow like plumbing, the larger and older the house, the more it leaks. For those new to consulting, get ready to start digging for the data. Analysts have their own hazing stories: late nights typing shipping data from paper invoices into spreadsheets, or consolidating data from 60 separate daily report emails into a single file. As long as companies defer the effort to clean up, they will continue paying premium hourly rates for mundane data collection.

Why Clients Have Data Problems

The first cluster of reasons centers on IT. Too often, IT makes only tactical repairs and spends its energy and budget playing catch-up. Clients customize their Enterprise Resource Planning (ERP) systems to match their processes instead of listening to systems integrators and sticking with best practices. The phrase we like to do it our way is usually code for messy data down the road.

The second cluster involves legacy thinking. We are all creatures of habit, and so are organizations. The phrase that is just the way we do it, or its corollary we tried that before but it failed, signals trouble and poor planning. There is always someone who knows how things really work. When that person leaves, the organization faces a quiet disaster. 2 outlines how profiling, standardization, deduplication and validation form the backbone of any serious data quality effort. A customer master file needs to be clean because it drives billing, accounting and customer relationship activities. Too often the same retailer appears as Wal-mart, Wallmart, Walmart and Wall Mart, and junky data makes analysis impossible.

The third cluster concerns roles and responsibilities. If data quality is everyone's job, it is effectively no one's job. It is not a good sign when veteran office workers are uncomfortable using basic spreadsheet commands like sort and pivot. Real-time analysis is sometimes not valued enough to appear in job descriptions and performance reviews, and that gap is ultimately the manager's fault.

The Time Split

From experience, the time allocation on a data-heavy project follows a consistent pattern. Collecting data consumes roughly one-third of the time. Cleaning and validating data takes another third. The actual value-added analysis, the real work the client hired you to do, gets the remaining third. Planning around this split prevents the false optimism that dooms timelines and frustrates clients.

Insights Matter, Not Data

Data is a bit of a misnomer because data alone is useless. It is like having a bag of flour, water, yeast, sugar, salt, butter and eggs in random portions and random containers. Good luck making bread. The progression from raw material to value follows a hierarchy. It starts as noise, gets organized into data, then turns into information as it is structured, cleaned and sorted so it makes sense. Analysis takes shape as information is pivoted, correlated, appended and hypothesis-tested. Insights are the gems and diamonds, rare and valuable and often polished. 3 formalizes this progression from raw data to information to knowledge to wisdom. Only analysis and insights should reach the client.

Know What You Are Looking For

Too often, consultants are unclear about what they are trying to prove. They drop a large, brutish and onerous data request that ineloquently says we will take everything, give us whatever you have. Instead, a professional forms hypotheses about what the answer could be. Working from right to left on the value hierarchy, the consultant reverse-engineers the analysis, the information and ultimately the specific data needed. This targeted approach saves time and signals competence.

Sometimes consultants must uncover, create, cleanse, triangulate or even fabricate data to answer key questions. Creativity is needed here, and it is a proven way to impress the client. The consulting toolkit for finding new data includes surveys, interviews, focus groups, workshops, financial comparisons, observations, estimates, simulations, business models, benchmarks and maturity models. The consultant who arrives with multiple sourcing strategies earns trust faster than the one who submits a single sprawling request and waits.

• 1Seth Godin on solving the perfect problem
• 2Master data cleansing practices that address customer record quality
• 3The DIKW pyramid, which models the hierarchy from data to wisdom