Measure First, Then Teach
Our Approach to AI Fluency, and What We Built to Support It
In my last post I argued that the major AI fluency frameworks are incomplete rather than wrong. They define the problem well and the person poorly. One thing up front. This piece describes something we have built, so treat it accordingly. I have tried to keep it closer to an honest account than a brochure, which is why there is a section near the end on what it doesn’t do yet.
The decision everything else follows from
Most AI upskilling programmes start with a curriculum. Ours starts with a baseline.
That sounds like a minor question of sequencing. It is not. If you start with a curriculum, you have decided what people need before you know anything about them. Every downstream choice inherits that assumption, and at the end of the programme you have no credible way to say whether anything changed, because you never established where anyone began.
The evidence on what that produces is not subtle. MIT’s Project NANDA, in The GenAI Divide: State of AI in Business 2025, found that 95% of generative AI pilots delivered no measurable impact on the profit and loss account, despite somewhere between $30 and $40 billion in enterprise spending. The headline figure has been contested since publication and I would not build an argument on it alone, but the report’s diagnosis is the useful part: the barrier to scaling was not infrastructure, regulation or talent. It was a learning gap. The systems, and the organisations around them, did not retain feedback or improve over time.
The pattern on the training side looks the same. Programmes report completion rates and satisfaction scores. Neither predicts whether a single person changed how they work on Monday.
Writing in HR Executive in June this year, Jill Barth opened a piece on new benchmark data with a line I have quoted more than once since: “Organizations that are pulling ahead on AI capability are the ones that measured first.” ServiceNow was the example. They assessed all 30,000 employees by job and by level, set percentile targets for each capability, and then gave every employee transparent access to their own scores and a personalised development path. Their Chief People and AI Enablement Officer, Jacqui Canney, described the design intent plainly: “We didn’t make it a stick. It was more like an incentive.”
That is the shape of the thing. Measure, then teach, then measure again with the same instrument.
Why we never ask anyone to rate themselves
There are no self-rated one to five scales anywhere in our assessment. Not one.
Workera’s 2024 analysis of more than 22,000 assessments, in which users predicted their own level before sitting an adaptive test, found that 71% could not accurately judge their own ability. Roughly a quarter overestimated themselves and nearly half underestimated. Fewer than three in ten got themselves right.
The detail that matters most for our purposes is where the error was worst. In hard technical domains such as machine learning, people were roughly twice as accurate. In the softer, fluency-adjacent domains, accuracy collapsed. On communicating about AI, 69% underestimated themselves and only 27% were accurate. Self-assessment is least reliable in exactly the territory that AI fluency occupies.
Workera’s 2026 enterprise benchmark, drawn from 88,753 assessments, makes the same point from the other direction. What a person can demonstrate and what they report about themselves are two different measurements, and course completion data is a record of attendance.
So we ask behavioural questions instead. What do you actually do, with which tool, on which piece of work, how often, and what happens when the output is wrong. Every question and every option is written around the client’s own tools, teams and document types, because an option list that could belong to any company produces answers that mean nothing.
Behavioural self-report is still self-report, and I am not going to pretend otherwise. The difference is that it is checkable against artefacts, and we check it. Each person’s real work is read alongside their answers: a proposal, a model, a deck, a report. Answers alone are a weak signal. Answers read against the work someone actually produced is a different proposition.
Fluency is role-relative, and nobody gets pre-labelled
The same behaviour means different things for a bid manager and a market analyst. So the rubric describing what each level looks like is written per role, in that role’s own workflows and tools, and it is agreed with the client sponsor before anyone is assessed. Per-role, per-dimension thresholds are generated alongside it.
There is a hard rule underneath that, and we enforce it in the software rather than leaving it to the facilitator’s discipline: the design phase never assigns a current maturity level to anyone. A role carries a target and nothing else. Current level is an output of assessment, never an input to design.
The reason is commercial as much as methodological. A baseline that was partly decided before anyone was assessed cannot afterwards be used to prove the programme worked. Sponsors notice this and that is how it should be.
Six dimensions, kept deliberately separate
Every gap we identify maps to exactly one of six dimensions: AI Literacy and Governance, AI Tooling and Workflow, AI Prompting Quality, AI Data Governance, AI Collaboration, and AI Business Impact.
They span the arc from knowing, through doing, to doing safely, to scaling it to others, to proving it was worth anything. We keep them separate because a single composite score hides the combination that should worry you most.
Consider what that combination looks like in practice. Verizon’s 2026 Data Breach Investigations Report found that 45% of employees are now regular AI users on corporate devices, up from 15% a year earlier. Cyberhaven’s 2026 AI Adoption and Risk Report found that 39.7% of all AI interactions involve sensitive data: source code, customer personal data, financial records, legal documents. A person can be genuinely excellent at prompting and simultaneously a live compliance exposure. A single headline number averages that person into the middle of the pack. Tagging every gap to one of six named dimensions is what surfaces the combination instead of smoothing it away.
Five Levels, named for people
Reserved, Responsive, Refined, Resilient, Radiant. Zero to four.
Level zero exists on purpose. Most frameworks begin at something like “novice user”, which quietly assumes the person has already started. Reserved is for the person who has not, whether through fear, distrust, or simply never having been given a reason. What they need first is psychological safety and a low-stakes first interaction, not a prompting workshop.
At the other end, Radiant is not about individual skill. It is about influence: contributing to organisational norms, leading on governance, mentoring. We identify those people from evidence during assessment rather than by asking for volunteers, and they become the internal network that carries the programme after we leave.
The Platform: Keystone People Intelligence
We built Keystone People Intelligence because the alternative was a consultant’s head and a folder of Word documents, which does not last beyond that consultant’s departure and does not scale from ten people to hundreds. It runs in four phases. Each has a distinct primary user, a distinct output, and a hard dependency on the phase before it.
Requirement. The programme is designed with the client sponsor in the room, across focused steps covering their business context, executive pain points, roles and headcount, programme goals, real use cases, maturity targets, apart from other areas. The output is a designed programme and a signed-off per-role rubric, produced before a single employee is contacted.
Review. Employees answer a questionnaire generated from that context, through a private link. Each person sees the core sections plus their own role’s questions, never every role’s questions. Each submission is assessed against the rubric, from evidence. The output is a fluency result per person and a set of organisation-level deliverables: the fluency baseline, an opportunity map, a prompt playbook, an ROI qualification, a training materials library, and the champion network.
Reform. Training is delivered and tracked to the individual gap it exists to close. Trainers get session guides tied to the client’s discovered use cases rather than generic lesson plans. Employees get their own portal showing their gaps and their assigned modules and their progress, and champions get their mentees. Line managers get a team view without being handed administrative rights.
Regulate. Report out, govern, prove movement, re-assess. Same instrument as the baseline, in a before-and-after framing.
Nothing is generic and nothing is thrown away. The context captured on day one shapes the questionnaire, which shapes each assessment, which shapes cohort assignment, which shapes what a trainer teaches on Tuesday, which shapes the report the stakeholder reads at the end. A gap identified in Review becomes a module assignment, becomes a completion record, becomes a closed-gap statistic in the final report.
The design choices I would defend
A few of these are worth naming, because they are where the opinions live:
Every gap owns a module. Each training module declares which gaps it closes, and coverage is calculated. A critical gap with no module attached is published on the plan as a defect. A gap nobody is going to act on is a planning failure, and it should be visible as one at assessment time rather than discovered at the end.
Cohorts are paced by their furthest-behind member. A cohort scheduled around its mean leaves its weakest members behind, and those are precisely the people the programme exists to move. We also check fit on two independent axes, level and availability, because someone who misses half the sessions will not reach the target regardless of where they started.
Engagement rosters sort worst-first. Stalled, then slowing, then not started, then active. The people who need intervention appear at the top of the screen rather than on page three.
The platform owns the arithmetic. AI writes the narrative in every report and nothing else. Averages, medians, role breakdowns and level distributions are computed from the actual result set and written over whatever the narrative proposed. Where the underlying data does not exist, the dashboard reports the number as unavailable. A governance dashboard that invents a figure is worse than one that admits it has none.
Everything leaves as a file. Questionnaires, rubrics, reports and full project checkpoints export to Word, Excel and PowerPoint. The client owns their artefacts whether or not they keep using us.
People are grouped by where they are, not by where they sit. The five cohorts map to the five levels rather than to the org chart. A Reserved person in a room of Resilient people learns nothing and leaves feeling worse. Fit is checked on two axes, level and availability, and the level band is calculated with the person's own score excluded, so their result cannot widen the range and hide their own mismatch. Anyone flagged gets a recommended better fit among the cohorts that actually exist.
Progress is made visible, which is the useful part of gamification. Each completed module earns points against an assigned maximum. Every gap carries a live status: closed, in progress, open, or unaddressed. Momentum comes from days since a person's last real signal, so someone reads as active, slowing, stalled or not started rather than as a binary tick. Certificates are uploaded and verified. There is a leaderboard for organisations that want visible recognition, and it stays optional, because public ranking tends to demotivate exactly the Reserved cohort the programme exists to move. Points measure completion rather than capability. Only the re-assessment proves movement.
Champions are identified from evidence, not from a volunteer list. The assessment flags people whose work already shows them influencing others, which is a different population from the top scorers. Each gets a defined role: peer trainer, use-case scout, governance lead or prompt curator. Each carries named mentees and a meeting cadence. Together they become the client's AI Radiant Network, and that mentoring layer is what lets a programme reach three hundred people without three hundred consultant hours.
All of this points at one outcome. The prompt library and case-study library grow from the client's own logged outcomes. Governance holds a named reviewer inside the client organisation, a scheduled re-assessment date and a review cadence. Everything exports as Word, Excel and PowerPoint. Most consulting engagements are quietly designed so the client needs the next one. The test I would apply to any AI fluency programme, including ours, is whether the client can run round two without the people who sold them round one.
How this answers the gaps in the last post
The frontline worker. New research from Workvivo by Zoom, covering 4,736 frontline and desk employees, found frontline workers three times less likely to use AI regularly. The cause is not access. Only 28% of frontline workers say their employer encourages AI use, against 55% of desk workers. Roughly three in ten get clear guidance, against half of desk workers. Separately, The Josh Bersin Company puts annual spend on frontline training, management development and career growth at around $400 a head, against more than $1,500 for white collar roles.
We have not solved this, and I want to be careful about what we do claim. Three specific things. The questionnaire downloads to Word or Excel, gets completed on paper or offline, and re-imports, so it works on a shift, on a site with no connectivity, and in a facilitated workshop. An administrator can record a work transcript for anyone who cannot upload a file, and that transcript is sufficient evidence on its own. And roles are the client’s actual roles, so nobody is assessed as a “general employee” alongside a marketing director.
Participants never get a corporate login. A private link during assessment, an access code afterwards. Both scoped to one project and one person. This matters more than it sounds: requiring corporate single sign-on quietly excludes contractors, agency staff, and much of the deskless workforce.
Differently abled and neurodivergent participants. No timed assessment. No video-only delivery. Answers autosave as drafts so the questionnaire can be done in pieces. Somebody else can record your evidence for you. This is a floor rather than a finished answer, and I would rather say so than claim otherwise.
Ethics as a mindset rather than a module. Data governance is one of the six scored dimensions. It can generate a critical gap that invalidates a training plan until a module addresses it. That is a materially different thing from an ethics module you tick.
The manager layer. Line managers get their own workspace with their own team’s data, because 91% of individual contributors saying their manager is not prepared to guide them is not a training problem you fix by training the individual contributors harder.
So what?
The frameworks were built in good faith by people working within the contexts they knew, so the gap has never really been one of intent. It sits in the distance between defining what fluency is and building something that measures it for a specific person doing a specific job in a specific company, then proves whether they moved.
That is the problem we decided to work on. Whether we have solved it is for clients to say rather than for me.
If you are running an AI upskilling programme, or about to sign one off, and you cannot currently answer the question “what would we measure at the end to know this worked”, that is worth a conversation. I am happy to walk through how we build a per-role rubric, what a credible baseline requires, and how you get to a champion network that keeps it running. None of that is proprietary.
Sudarshan has spent two decades in enterprise transformation, strategy and governance, working across platforms and industries to accelerate tech modernisation for global clients. Born in India and formerly Global CTO and Cloud Lead at a major consulting firm, he specialises in data and AI, and in having the difficult conversations clients usually avoid, especially the ones about what not to do.
References
1. MIT Project NANDA (2025). The GenAI Divide: State of AI in Business 2025. Covered in Fortune and Forbes, August 2025. The 95% figure has been publicly disputed on methodological grounds.
2. Barth, J. (2026). Verified AI skills lag far behind what employees self-report, new data finds. HR Executive, 4 June 2026. hrexecutive.com
3. Workera (2026). 2026 AI Skills Enterprise Benchmark Report. 88,753 assessments across 32,422 individuals and 14 capabilities. workera.ai
4. Workera (2024). 7 out of 10 employees dangerously underestimate or overestimate their skill levels. Analysis of 22,000+ domain assessments through early 2024, first published on Fortune.com, 7 October 2024. workera.ai
5. Verizon (2026). Data Breach Investigations Report 2026. verizon.com/business/resources/reports/dbir
6. Cyberhaven (2026). AI Adoption and Risk Report 2026, 11 February 2026. cyberhaven.com
7. Workvivo by Zoom (2026). The Frontline AI Gap. Study of 4,736 frontline and desk employees. Reported in Customer Experience Magazine, 18 June 2026.
8. The Josh Bersin Company (2026). Frontline workforce investment analysis and frontline workforce taxonomy. joshbersin.com
9. Acorn (2026). 2026 State of Learning for AI Fluency Report. Survey of 1,200+ professionals across the US, Canada and Australia, fielded April 2026.
