| AI Adoption in Organisations and Government Five structural concerns for strategy, capability, and accountability A briefing for the British Computer Society David Sutton CITP MBCS · Southport Innovation Centre · May 2026 |
This briefing sets out five structural concerns about the adoption of AI across organisations and government. The concerns apply wherever AI is being used for analysis, advice, briefing, or strategic decision-making. Government represents the most acute case — the stakes are highest and the consequences of getting it wrong are least correctable by market forces. But the underlying questions belong to every organisation making significant AI adoption decisions.
The briefing does not take a political position. It addresses questions of capability, sovereignty, and accountability that apply regardless of which administration is in office, which sector is involved, or what reform programme is being pursued. The concerns are not theoretical. Several are already in motion.
| Why government is the acute case Private organisations face market correction when AI adoption goes wrong. Government does not have that feedback mechanism in the same way. When a commercial organisation loses analytical independence, a competitor gains advantage and pressure for correction builds. When a government loses it, the mechanism that would generate the pressure for correction is the same mechanism that has been impaired. The concerns are structurally identical across sectors. The consequences in government are harder to reverse. |
1. The narrowing of strategic thinking
The concern — applicable to any organisation
AI systems reflect the assumptions built into their design and training. When used for briefing, analysis, or strategic advice, they tend to return well-structured versions of questions already within their frame. This is not a defect — it is how the technology works. But it creates a specific risk wherever AI is used at the level where the quality of questions matters as much as the quality of answers.
The people whose primary function is to ask questions the prevailing frame has not yet formulated — strategy directors, senior advisers, board members, policy leads — are precisely those for whom this narrowing tendency is most consequential. If AI-assisted briefings progressively shape what questions seem worth asking, the range of thinking available to decision-makers narrows without anyone making that choice deliberately.
The narrowing is gradual and largely invisible. Each individual output appears comprehensive. The cumulative effect over months and years of sustained use is that certain kinds of question become progressively less available. This is not the AI providing wrong answers. It is the AI providing well-formed answers to a progressively narrower set of questions.
Why government is the acute case
In commercial organisations, strategic narrowing eventually shows in competitive performance. A competitor who asks better questions gains advantage; the feedback, though slow, exists. In government, the feedback loop between the quality of strategic questions and visible outcomes is longer, more diffuse, and more easily attributed to other causes. Narrowing can compound for years before it becomes visible as a problem rather than a background condition.
Ministers and senior officials who rely on AI-assisted briefings across multiple policy areas simultaneously are exposed to the narrowing effect across their entire portfolio, not just one domain. The diversity of challenge that healthy governance requires — different analytical lenses, different framings, genuine disagreement about what questions matter — is reduced structurally, not through any individual failure of judgment.
| Practical implication for all organisations AI adoption in advisory and analytical roles requires an explicit mechanism for maintaining the questioning that AI cannot generate. This is not an argument against AI in these roles. It is an argument for designing its use with the narrowing tendency explicitly accounted for. The mechanism does not need to be complex: it needs to ensure that the questions the AI did not raise are asked by someone, routinely, as a designed part of the process rather than an afterthought. |
2. The dependency cascade and loss of sovereign capability
The concern — applicable to any organisation
The dependency cascade describes a well-documented pattern in technology adoption. It proceeds in stages, each individually rational, whose cumulative effect is the progressive loss of the internal capability needed to govern what has been adopted.
| Phase | What is outsourced | What is lost |
| Phase 1 | AI tools adopted from external providers | Internal development capability |
| Phase 2 | AI-led transformation managed externally | Change management knowledge |
| Phase 3 | AI as licensed service, not owned capability | Ability to modify or exit |
| Phase 4 | AI strategy developed by external consultants | Strategic direction |
| Phase 5 | Direction shaped by provider capabilities and constraints | Sovereign judgment |
By Phase 3, the organisation cannot see what it has lost — because seeing would require the capability it no longer has. This is not hypothetical. It is the pattern that produced the Post Office Horizon disaster, repeated NHS IT failures, and Birmingham City Council’s financial collapse. In each case, the dependency was visible in retrospect. The warning signs were present throughout. The test for every significant AI procurement decision is straightforward: after five years of this contract, will we be more or less able to govern this system ourselves?
Why government is the acute case
Commercial organisations that reach Phase 4 or 5 face competitive disadvantage and, eventually, board-level pressure to rebuild internal capability. Government organisations face neither pressure in the same form. A department that has outsourced its analytical and strategic AI capability has also outsourced the capacity to recognise that this has happened. The dependency can persist indefinitely, sustained by the institutional memory that this is simply how things are done.
There is an additional dimension in government that does not apply in commercial contexts: supplier nationality. An organisation whose AI strategy is shaped by a provider headquartered in another country has introduced a dependency that is not only commercial but geopolitical. The provider’s home government may have legitimate or illegitimate interests in what the AI emphasises, excludes, or advises. This is not a hypothetical risk. It is already a live consideration in telecommunications infrastructure and is not yet being applied systematically to AI.
| The 70/30 threshold — applicable to all organisations, critical in government 70% of strategic AI and analytical capability should remain internal and domestically governed. 30% may be sourced externally where appropriate and non-critical. No critical function should be wholly dependent on a single external or foreign provider. For commercial organisations this is a strategic resilience question. For government it is also a national security question. The principle is the same; the stakes differ. |
3. The reversal of roles: from judgment to navigation
The concern — applicable to any organisation
There is a practical difference between using AI to extend judgment and using AI as the source of judgment. The first preserves the human’s analytical capability. The second progressively replaces it.
In the first mode, the person brings formed judgment, domain knowledge, and the capacity to evaluate what the AI produces critically. The AI extends the reach of that judgment across more information, more quickly. In the second mode, the person’s primary skill becomes working effectively with AI outputs — selecting, adjusting, and presenting what the system generates. The AI becomes the source of the thinking rather than an instrument for extending it.
The difference is not visible in normal operation. Both modes produce outputs. Both look productive. The difference emerges under pressure, in unfamiliar situations, or when the AI is wrong. A person operating in the first mode can identify where the AI’s reasoning fails and correct it. A person operating in the second mode cannot — and may not know they cannot until the moment when it matters.
Why government is the acute case
New ministers, new advisers, and officials rotating between departments arrive in post carrying general capability but limited domain knowledge. In previous eras, they developed domain knowledge through exposure to experienced officials and the accumulated institutional record. If that knowledge is now mediated primarily through AI systems, the new arrival is learning the domain through a filter that reflects the AI’s training rather than the department’s specific history, the complexity of its current situation, or the questions that experienced officials would know to raise.
The concern is not that AI briefings are inaccurate. It is that they are accurate at the wrong level — providing well-organised information about the domain as it is generally understood, rather than the particular, contested, historically-embedded reality of this department, this policy, this moment. The judgment needed to govern effectively requires the latter. AI currently provides the former.
| The practical test — for any organisation Can the person using this AI system articulate, without AI assistance, the reasoning behind the conclusion the AI reached — and identify where that reasoning might be wrong? If the answer is no, the AI is operating as the source of judgment rather than its extension. That is a capability risk in any organisation. In government it is also an accountability risk: ministers are accountable to Parliament for decisions they must be able to explain and defend. Decisions whose reasoning they cannot access independently cannot be defended independently. |
4. Planning for the future as it will actually be
The concern — applicable to any organisation
AI systems are trained on historical data. They reason well about patterns present in their training. They are structurally limited in reasoning about genuinely discontinuous futures — futures in which the patterns that structured the past no longer apply.
There is a discipline that addresses this directly: designing strategy from the future backward rather than from the present forward. The question is not what the current situation suggests about likely futures. It is: what must be true now for a specific, well-defined future to be reachable? That question produces a different set of strategic priorities from extrapolating forward from the present — and it is a question AI systems using historical pattern-matching are poorly positioned to lead.
The practical risk is that AI-assisted strategic planning produces rigorous analysis of futures that resemble the present, while the futures that actually require preparation are discontinuous with it. The analysis is thorough. It is aimed at the wrong target.
Why government is the acute case
Commercial planning horizons are typically three to five years, within which historical pattern-matching is reasonably reliable. Government infrastructure, legislative architecture, and institutional design operate across decades. The decisions being made now about AI adoption in public services, national security, justice, and healthcare will shape what is possible in 2035 and 2040. Those futures are not reliably predictable from patterns in historical data.
There is a specific compounding risk in government: AI systems shape the information environment in which planning takes place. If the analysis informing long-term policy is itself generated by systems trained on the recent past, the planning process is oriented toward what has been rather than what will be. The broken clock problem — designing for the world as it was understood at the moment of construction — is not just a risk of bad strategy. It is built into the analytical tools being used to develop the strategy.
| The clock-setting question — for any organisation Every significant AI adoption decision is a clock-setting decision. The architecture chosen now determines what is possible later. For commercial organisations, poor clock-setting produces competitive disadvantage that can be corrected over years. For government, architecture decisions about AI in public services, national security, and democratic infrastructure compound across decades and are very much harder to reverse. The principle — design from the future you want, not the present you have — applies in both cases. The cost of getting it wrong differs substantially. |
5. Accountability and the information environment
The concern — applicable to any organisation
Accountability depends on a functional feedback loop between reality and the people responsible for governing it. In commercial organisations, this loop runs between market outcomes, board oversight, and management decisions. In democratic systems, it runs between citizen experience, parliamentary scrutiny, and government decisions. In both cases, AI operating at scale in the information and analytical environment can disrupt this loop in ways that are difficult to detect and slow to become visible.
The mechanism is specific. If the analysis reaching decision-makers is generated by systems with similar training, similar tendencies, and similar blind spots, the diversity of perspective that good governance requires is reduced — not by intention, but by the structural properties of the technology. Boards that rely heavily on AI-assisted management information, and governments that rely heavily on AI-assisted policy analysis, may find that challenges to the prevailing view become progressively harder to surface, not because they are suppressed but because the analytical infrastructure does not generate them.
There is a harder version of this concern that applies particularly in democratic contexts. AI systems optimised for engagement rather than accuracy can make poor outcomes feel like good ones, manufacture apparent consensus where none exists, and replace evidence-based assessment with sentiment-calibrated messaging. The technical capability for this exists now. Existing governance frameworks were not designed to detect it.
Why government is the acute case
Commercial organisations that lose the feedback loop between reality and governance face consequences through markets, regulators, and eventually through courts. Democratic governments that lose it face a more fundamental problem: the mechanism through which citizens hold government accountable is the same mechanism that has been impaired. The self-correcting function of democratic accountability requires that accurate information about outcomes reaches those with the authority and motivation to act on it. AI at scale in the information environment can disrupt this at precisely the points where it matters most.
The Post Office Horizon case is the clearest recent example of what happens when the feedback loop between reality and governance is severed at institutional level: accurate information about what the system was actually producing was present in the system, but could not reach those with authority to act on it. AI adoption that replicates this structure — generating confident outputs that are systematically wrong in ways that cannot be detected from inside the system — carries the same risk at larger scale.
| The structural gap — in all organisations, most critical in government No current AI governance framework includes an independent function with genuine authority, protected reporting lines, and a mandate that cannot be withdrawn for the discomfort it causes. The Data Protection Officer role under GDPR demonstrated what this looks like in practice and why it works. Until an equivalent exists for AI governance, ethics and accountability frameworks will measure the ethics that survives observation rather than the ethics the situation requires. This gap exists across sectors. In government, where the consequences of undetected failure are borne by citizens who had no part in the design decisions, it is most urgent to close. |
Questions any serious AI adoption programme should answer
These questions apply to any organisation making significant AI adoption decisions. They are stated in terms applicable to both commercial and government contexts. The absence of a clear answer to any one of them is itself important information about the state of the programme.
| For any organisation | Additional dimension in government | |
| 1 | After five years of each major AI contract currently being signed, will we be more or less able to govern the system ourselves? Is this question being asked before contracts are signed? | Does the contract introduce a dependency on a provider whose home government has interests that may diverge from the national interest? Is this being assessed? |
| 2 | What mechanism exists to ensure decision-makers are developing the judgment to evaluate AI outputs critically — rather than the skill to navigate AI outputs efficiently? | Can ministers explain and defend AI-assisted decisions to Parliament without access to the AI? If not, the accountability requirement is not being met. |
| 3 | What proportion of strategic analytical capability is internally held? Is this being tracked, and against what threshold? | What proportion of national security, policy development, and ministerial briefing capability depends on foreign commercial providers? Is there a sovereign floor? |
| 4 | Where AI is used for analysis and advice, what is the process for identifying the questions the AI is not raising — not the questions it answers poorly, but the questions it does not formulate? | Across departments using similar AI systems, is there diversity of analytical challenge or systemic convergence on similar questions and similar framings? |
| 5 | Is strategy being designed from a defined future horizon backward — or from the present situation forward, using pattern-matching on historical data? | For decisions with decade-scale consequences, is there an explicit process for reasoning about discontinuous futures that historical AI training cannot reliably generate? |
| 6 | What independent function, with genuine authority and protected reporting lines, is responsible for examining what AI adoption is doing to the quality of decision-making over time? | What independent function is examining what AI adoption is doing to the functional feedback loop between citizen experience and democratic governance? |
What this briefing is not saying
This briefing is not arguing that organisations or governments should adopt AI more slowly, or that AI is unsuitable for analytical, advisory, or strategic roles. The case for AI adoption is substantial. The productivity gains, the analytical reach, the capacity to process information at scales impossible for human teams — these are real.
The argument is specific: the structural concerns set out above are not currently being examined with the same rigour as the capability and efficiency questions. And the consequences of not examining them compound over time in ways that are difficult to reverse.
AI amplifies what it is given. Introduced into an organisation that has preserved its analytical independence, its sovereign capability, and its mechanisms for genuine accountability, it amplifies those things. Introduced into an organisation that has not, it amplifies their absence. The difference is not visible at the point of adoption. It becomes visible later, under pressure, when correction is most difficult.
The question is not whether to adopt AI. It is whether the conditions that make AI adoption beneficial rather than harmful are being built at the same pace as the adoption itself. In commercial organisations, the market will eventually surface the answer. In government, there may be no mechanism to surface it at all if the adoption decisions now being made are not made with this question in view.
David Sutton CITP MBCS · Southport Innovation Centre · May 2026