Post 11 of 19
Why most GenAI pilots fail, and what AI-ready data actually looks like
Every enterprise leadership team is having some version of the same conversation. Where do we deploy AI, how fast can we move, what is the competitive cost of waiting? It is a reasonable question, but it usually skips the prior question: is our data in any state for AI to work? Research keeps pointing at the same bottleneck. RAND's 2024 study of why AI projects fail names a lack of adequate data among the leading root causes, and notes that by some estimates more than 80% of AI projects fail — roughly twice the rate of IT projects that do not involve AI. The pattern is consistent: projects that fail often fail at the data layer, not the model layer. LLMs amplify the problem because they are confident by default — fed bad data, they produce articulate, plausible, and wrong answers, often at a speed that outpaces any human ability to verify them. This post lays out what AI-ready data actually looks like, why most organisations are not there yet, and how to close the gap pragmatically.
The old data-quality maxim has not changed; the consequences have. When a flawed dataset fed a traditional BI dashboard, an analyst could catch the anomaly and flag it. When the same dataset feeds an LLM-powered assistant or a recommendation engine, the bad output gets surfaced directly to a customer, a regulator, or a decision-maker with no intermediate check. The model does not know the data is wrong. It will defend the answer eloquently. And because LLMs are trained to be helpful and confident, they rarely volunteer the kind of caveats a human analyst would attach to a shaky number. The result is an entire new category of risk: at-scale, automated, articulate hallucination. The mitigation is not better prompting or guardrails. It is clean, well-governed, well-labelled input data.
80%+
of AI projects fail by some estimates — about twice the failure rate of IT projects without AI
Source: RAND Corporation, 2024 ↗30%
of generative AI projects predicted to be abandoned after proof of concept by end-2025, with poor data quality among the causes cited
Source: Gartner, 2024 ↗Large language models are extraordinary pattern recognisers, but they have specific failure modes that interact badly with messy enterprise data. They tend to interpolate when data is missing rather than flag the gap. They will confidently summarise a contradictory dataset by averaging the contradictions away. They will treat schema variations as semantic equivalences ('customer_id' and 'cust_no' must mean the same thing — except sometimes they do not). And they will hallucinate plausible field values when asked to fill in incomplete records. None of these behaviours are bugs; they are the natural outputs of a system optimised for fluency. The only durable fix is to give the model inputs that do not require it to interpolate, contradict, or hallucinate in the first place.
Pre-AI Data Issue
Duplicate customer records caused inflated counts in reports
Inconsistent unit conventions caused metric drift between departments
Missing values were caught and flagged by analyst review
Stale data showed up as obvious outliers in trend charts
Ambiguous metric definitions caused intermittent meeting debates
Post-AI Amplification
AI assistant gives different customer counts depending on which dedup logic it picks
Model averages tonnes and kilograms invisibly, producing fluent nonsense
Model interpolates missing values without flagging the gap to the user
Stale data is summarised as current, with no temporal caveat
Different users get different definitions from the same assistant on the same day
AI-ready data is not exotic. It is the same clean, governed, well-modelled data that produces good BI — held to a slightly higher standard on a few specific dimensions. The bar rises on completeness, semantic clarity, and metadata richness because models cannot rely on human context to fill the gaps. Below is a practical checklist we use with clients evaluating their AI readiness before any pilot work begins.
Single source of truth: every key entity (customer, product, transaction) has one canonical representation, with deduplication and identity resolution applied
Schema discipline: stable, documented schemas with explicit field names, units, and value domains — no surprises hiding in free-text fields
Completeness baselines: every critical field has a measured completeness rate, and gaps below threshold trigger alerts
Semantic layer: business terms have one definition, mapped explicitly to the underlying SQL, so the model and the human see the same metric
Lineage and audit trail: every output can be traced back to source records — non-negotiable for any AI feature touching regulated decisions
Sensitivity classification: PII, commercial confidential, and restricted fields are tagged at the row or column level, so AI features can enforce access policy without leakage
Sufficient history: enough clean history to support fine-tuning, retrieval, and trend reasoning — typically 18–36 months for operational use cases
The pattern of AI pilot failure is depressingly consistent. A senior leader champions a high-visibility use case. A small team builds a demo on a curated subset of data. The demo dazzles in a steering committee. The team then tries to extend the same approach to the full production data estate, discovers that the underlying data is far messier than the curated demo set, and the project stalls in an extended remediation phase that nobody budgeted for. By month nine, the political appetite has moved on. The pattern that works is the opposite: assess data readiness first, fix the foundation in parallel with a small narrow pilot, and only then scale. Less dramatic, much higher hit rate.
A useful diagnostic question for any proposed AI use case: what dataset does this depend on, and would I trust that dataset to inform a quarterly board decision today? If the answer is no, the AI use case is not ready — the data is the project.
The sequence matters. Most organisations get more value from spending a quarter on data readiness and a quarter on a narrow pilot than from spending six months on an ambitious pilot built on shaky foundations. The order of operations is the difference between AI as a strategic capability and AI as an expensive series of demos.
Pick one high-value use case with measurable economics — not the most ambitious, the most measurable
Audit the underlying data for that specific use case against the AI-ready checklist; remediate before the pilot
Build the pilot narrowly, with explicit ground truth for evaluation, and run it in shadow mode against existing processes
Measure rigorously: accuracy, latency, cost per query, user trust scores — not just sentiment from the executive sponsor
Only after the narrow case is robust, generalise the platform investments — vector store, governance, observability — to the next use case
Delta BI runs this readiness work as the precursor to any AI engagement. The most expensive AI mistake is not picking the wrong model; it is deploying the right model on the wrong data.
The bottom line
AI is not a substitute for a working data foundation; it is the most demanding consumer of one. Models amplify the quality of their inputs in both directions — clean data produces step-change value, messy data produces confidently wrong answers at industrial scale. The organisations winning with AI are not the ones with the largest model budgets. They are the ones who did the unglamorous work of cleaning, governing, and modelling their data before the pilots started. Delta BI builds that foundation: the warehouse, the governance, the semantic layer, the observability — the layer that makes AI useful instead of merely impressive. The data work is the AI work.
Start with the decision
Tell us about your challenge. We will reply to arrange an assessment and identify a practical route to delivery.
Request a data assessment