Patrick Rose is a sessional academic at Griffith University in Australia, whose academic work involves digital forensics and cybersecurity investigation. With a professional background spanning cybersecurity, enterprise IT and risk management, Rose argues that organisations can become increasingly effective at satisfying measurable targets while progressively losing the human judgement those targets were originally intended to support. Full bio at the end of the article.
Key Takeaways
Better metrics do not guarantee better audits. Improving indicators may conceal declining professional judgement.
Judgement Displacement Risk can arise without misconduct. Well-intentioned firms may prioritise measurable targets over judgement.
AI could amplify these risks. Efficiency gains may weaken critical thinking, junior training and effective review.
Human oversight must be meaningful. Auditors must be able and empowered to challenge AI-generated work.
Regulators must look beyond dashboards. Strong quality-control metrics do not necessarily prevent audit failures.
Audit firms have never had more tools for measuring quality. Compliance dashboards, inspection results, remediation programmes and increasingly sophisticated monitoring systems are designed to provide assurance that professional standards are being maintained. But what if improving performance indicators conceal a deterioration in the very judgement those systems are intended to protect?
It is a question with growing relevance as the Big Four accelerate their adoption of artificial intelligence. AI promises substantial productivity gains, and audit clients are already pressing for a share of those efficiencies. Yet the ability to produce work more quickly does not necessarily mean that the work is being assessed more critically.
The risks are already visible in AI-generated errors and fabricated references in reports produced by Big Four firms. Former global CFO Wayne Banks has also warned about automation deference — the tendency to place excessive trust in apparently authoritative AI output. Meanwhile, the growing use of automation raises questions about how younger auditors will acquire experience when the routine work traditionally used to develop professional judgement is increasingly performed by technology.
As the US Public Company Accounting Oversight Board considers moving towards a more quality-control-focused inspection model, Rose raises a fundamental question: can an organisation become better at demonstrating that its controls are working while becoming less capable of recognising what those controls fail to detect?
In this Expert Voices interview, Rose applies his concept of Judgement Displacement Risk to audit quality, exploring the limitations of performance metrics, the risks of AI-driven efficiency, the future of the traditional audit pyramid and what meaningful human oversight requires.
The Interview
What Is Judgement Displacement Risk?
Your research develops the concept of Judgement Displacement Risk — where organisations become increasingly successful at satisfying measurable proxies while losing the judgement those measures were meant to support. What first led you to identify this as a distinct governance risk?
It started somewhere rather unexpected. I became interested in how information loses qualification as it becomes easier to communicate and operationalise — initially through semantic bleaching, Zipf’s Principle of Least Effort and the truncated saying, “a jack of all trades is a master of none.” That eventually led me to a broader question: whether organisations do something similar when they simplify reality into measures that can be managed. JDR emerged from investigating what gets lost in that process.
Can Audit Quality Improve on Paper While Judgement Deteriorates?
Audit firms rely heavily on methodologies, checklists, quality indicators and remediation metrics. Could an audit firm appear to be improving — with greener dashboards and better compliance scores — while professional scepticism and judgement are actually deteriorating underneath?
Yes. That is almost the canonical JDR problem. Dashboards can represent compliance, completion, remediation and other observable indicators, but professional judgement and scepticism are only partly observable through such measures. A firm can therefore improve everything it measures while becoming less capable of seeing what those measures omit.
The warning sign is not that the dashboard is green. It is that the organisation increasingly treats the green dashboard as sufficient evidence that the underlying reality is also healthy.
Is It Gaming the System — or Something More Fundamental?
You argue that fewer escalations are not necessarily evidence that a system is becoming safer. How does “gaming the system” fit into Judgement Displacement Risk, and what should leaders look for when trying to distinguish genuine improvement from apparent improvement?
Gaming is related, but it is not the same thing.
Goodhart’s and Campbell’s observations describe what can happen when people respond strategically to measures and incentives. JDR does not require anybody to game anything. The organisation can sincerely optimise its processes and still progressively select for what is measurable while selecting against activities — including contextual judgement — that are harder to represent.
Leaders therefore need to ask two questions: Are the measures still valid proxies for what we actually care about? and What important things have become less visible because they are not represented by those measures?
Could AI Make a Bad System More Efficient?
One of the claims in your research is that, for a time, a bad system can actually be more efficient than a good one. The Big Four are under growing pressure to translate AI into faster work and lower costs, and audit clients are already seeking fee reductions based on expected AI efficiencies. Does that create exactly the kind of environment in which JDR can develop — where productivity improves faster than our ability to tell whether judgement and quality are being preserved?
Yes. The danger is that productivity is comparatively easy to measure while preservation of judgement is not.
A system can therefore become faster, cheaper and apparently more efficient while simultaneously becoming more brittle. The distinction I draw from Kant is between applying an existing category or rule and recognising when the situation requires reconsidering the category itself. Automation is exceptionally good at the former. Professional judgement becomes critical at the boundary between the two.
That means efficiency can improve faster than our ability to determine whether the system remains capable of handling the exception.
Is Automation Deference Making the Problem Worse?
Wayne Banks has warned in Big4News about “automation deference” — the risk that polished, apparently authoritative AI output makes people less inclined to question or challenge it because it looks competent. Does that fit within your concept of Judgement Displacement Risk? As the Big Four expand their use of AI, how do firms stop human judgement from becoming a rubber stamp on increasingly persuasive machine-generated output?
It is closely related but analytically different.
Automation deference or automation bias concerns the human tendency to give undue weight to an apparently authoritative machine output. JDR concerns what happens when the surrounding organisation is progressively designed around what the automated system can represent and optimise.
The two can reinforce each other: the system reduces opportunities to exercise judgement, while increasingly persuasive AI makes people less inclined to use the opportunities that remain.
What Does Meaningful Human Oversight Require?
Professional-services firms increasingly emphasise that humans will remain “in the loop” as AI is deployed. But what does meaningful human oversight actually require? Is a human really exercising judgement if they lack the authority, information or organisational confidence to override the system?
A human being “in the loop” is meaningless if that person lacks the information, time, authority or organisational protection required to disagree.
Meaningful oversight requires a genuine capacity to question the model, inspect the underlying evidence, halt or override its action and escalate something that does not fit the established categories. Otherwise the human may be present procedurally while judgement has already migrated elsewhere.
Could AI Undermine the Development of Future Auditors?
EY US has recently announced a programme rewarding human capabilities including judgement and adaptability.
Separately, KPMG and the University of Texas at Austin studied 523 early-career KPMG professionals using an AI agent on realistic work tasks. Compared with the AI working alone, 50.1% of participants improved the output, 25.8% added little or no value, while 24.1% performed worse than the AI working alone.
Traditional measures such as domain knowledge, critical-thinking ability and AI literacy did not clearly predict which group someone would fall into. What do these findings tell us about the cognitive risks of introducing AI into professional work? And as AI takes over more of the routine work through which junior professionals traditionally built experience, could firms inadvertently weaken the development of the judgement they will later need to supervise AI effectively?
They suggest that simply putting a knowledgeable professional beside an AI system does not guarantee that the combination will outperform the AI alone. Human contribution depends upon something more subtle than conventional measures of expertise or AI literacy.
The longer-term risk is developmental. Junior professionals traditionally acquire judgement partly through repeated exposure to routine cases, mistakes, anomalies and senior correction. If AI removes too much of that apprenticeship layer, firms may eventually discover that they have automated away part of the process by which future reviewers learned what deserves questioning.
The objective should therefore be augmentation rather than cognitive substitution: AI can provide expertise and analytical capacity, but the professional must still develop the ability to evaluate, synthesise and challenge it.
Can the Audit Review Pyramid Keep Up With AI?
There is also a separate quality-control problem. The Big Four model relies heavily on junior staff producing detailed work for successive layers of senior review. AI can now produce far more work, far faster — including plausible but hallucinated sources, citations or unsupported conclusions — while senior managers and partners have limited time to interrogate the underlying output. Does the traditional review pyramid still work in that environment? What needs to change to prevent AI-driven productivity from overwhelming firms’ capacity to verify what they produce?
Not unchanged.
The traditional pyramid assumes that the volume of material produced at junior levels remains broadly compatible with the reviewing capacity above it. AI breaks that assumption: generation can scale much faster than human verification.
The answer cannot simply be to generate more and review it in the same way. Firms will need stronger provenance, verification and exception mechanisms, potentially including machines checking machines, but ultimately someone must remain accountable for the evidential basis of the work. AI can accelerate production; it cannot eliminate ownership of the conclusion.
How Can Firms Keep Autonomous AI Under Control?
Recent frontier-AI evaluations have demonstrated how highly capable models can behave in unexpected ways when given greater autonomy, tools and system access. At the same time, the Big Four are giving AI systems access to increasingly large volumes of sensitive client information. From a JDR and cybersecurity perspective, how should firms design these systems so that unexpected AI behaviour can actually be detected, challenged and stopped before it becomes consequential?
The central design principle is that capability should not imply authority.
AI systems should receive only the data, tools and permissions necessary for their task, with consequential actions separated by approval boundaries. Firms also need observability: logging, provenance, independent monitoring and the ability to reconstruct what the system did and why.
And there must be a genuine stopping mechanism. If neither the human operator nor an independent control can detect, interrupt or revoke an autonomous action, calling the arrangement “human oversight” provides very little assurance.
Data sovereignty and cross-border processing are additional concerns, but they are only part of the problem.
Could the PCAOB Miss Problems Hidden by Quality Metrics?
You are based in Australia, but in the US the Public Company Accounting Oversight Board — the regulator responsible for inspecting auditors of listed companies — has signalled a shift towards a more quality-control-focused approach to inspections. Through the lens of Judgement Displacement Risk, what risks does that create? Could a firm’s quality-control system and metrics look increasingly robust while problems in the judgement exercised on individual audits become harder for the regulator to see?
Potentially, yes.
A firm’s quality-control system could genuinely improve — better training, consultation, monitoring, remediation and internal inspection — while a particular engagement still contains a judgement failure that none of those systems recognises.
The JDR danger arises if evidence that the quality-control system is improving begins to substitute for independent observation of the audits it is supposed to improve.
The dangerous inference would be: “our declining deficiency rate proves the system is working,” when another possible explanation is, “we have reduced the measurement channel capable of discovering the deficiencies.”
Does Results-Based Consulting Create New Risks?
The consulting industry is under growing pressure to move away from time-based billing towards fees linked to measurable outcomes. Through a JDR lens, does that risk creating a new proxy problem — where consultants optimise their work around the measures that determine whether, and how much, they get paid? How can results-based pricing reward genuine value without encouraging firms to deliver the metric rather than the underlying outcome?
Potentially. It solves one proxy problem while creating another.
Time billing rewards an input — hours — that may bear little relationship to value. Results-based pricing moves the incentive closer to what the client actually wants. The difficulty is that the client’s real objective is rarely perfectly contractible, so payment normally attaches to a measurable representation of the outcome.
Once money is attached to that representation, the consultant has a rational incentive to optimise it.
The design challenge is therefore to ensure that the payable metric remains evidence of value rather than becoming the definition of value. That requires multiple safeguards, particularly around attribution, time horizons, unintended consequences and the continuing ability to exercise judgement when achieving the contracted measure would no longer serve the client’s underlying objective.
Further Reading
Research and regulatory sources
PCAOB: Amendments to Its Quality Control Standard — The regulator's September 2026 amendments to QC 1000, which it describes as supporting its proposed move towards quality-control-focused inspections.
PCAOB: Quality Control Standard QC 1000 — The regulatory framework governing audit firms’ systems of quality control, including monitoring, evaluation and remediation requirements.
Related Big4News coverage
AI Governance in Audit: Wayne Banks on Automation Deference — An Expert Voices interview examining the risks of excessive reliance on AI-generated output and the importance of preserving independent professional judgement.
What Arthur Andersen’s Training Model Can Teach the Big Four About AI — How automation threatens the traditional apprenticeship model through which junior auditors develop professional scepticism and judgement.
PwC Becomes Fourth Big Four Firm Linked to AI Hallucinations — Examines fabricated references and AI-generated errors, and why conventional review hierarchies may struggle to detect them.
KPMG US Redesigns Audit Training as AI Takes Over Routine Testing — Explores how the introduction of AI is reshaping audit training and the development of professional expertise.
KPMG Asked for the AI Dividend. Now Its Clients Want One Too — Examines the commercial pressure to translate AI productivity gains into lower audit fees and the implications for audit quality.
EY Is Spending $100 Million to Reward the ‘Human’ Skills AI Lacks — Examines EY’s approach to recognising human capabilities, including judgement and adaptability, as AI transforms professional work.
Who is Patrick Rose?
Patrick Rose is a sessional academic at Griffith University in Queensland, Australia, where his work involves digital forensics, evidentiary reasoning and cybersecurity investigation.
He has more than two decades of professional experience spanning information technology, cybersecurity, enterprise infrastructure and risk management. His career has included technical roles at IBM Global Services, EYI and GlaxoSmithKline, as well as developing secure communications systems and security infrastructure for government and financial-sector clients in the United Kingdom.
His current research focuses on Judgement Displacement Risk (JDR), a governance concept examining how organisations can become increasingly effective at satisfying measurable performance indicators while progressively undermining the human judgement those indicators were intended to support. His work explores the implications for artificial intelligence, organisational governance, professional expertise and the design of effective human oversight.
Rose is based on Australia’s Gold Coast.
About Claudine Cassar
I’m a corporate anthropologist and former Deloitte equity partner. I sold my technology business to Deloitte in 2016 and led the Malta Consulting team for five years. I am the founder and editor of Big4News, which provides independent, clear analysis of PwC, Deloitte, EY, and KPMG — free from corporate spin.
Find me on LinkedIn, X, Instagram, or my author website.
Feel free to reply to this newsletter — I read every reply.
An additional piece to this excellent expose of modern forms of decision-theory and various forms of displacement risk. Aristotle well before AI in founding western ethics argued as did later virtue ethicists such as Aquinas tht the key to acquiring wisdom and judgement was the practice of human skill (virtus) in an apprenticeship model by observing in action a practitioner and also in providing feedback to same.
I look at what's happening in the big 4 is the model has been upended. The senior partners are 'rain makers' and 'work getters', tap the work to the younger workers without adequate supervision and then let work go through to clients without quailty controls.
It's a departure from the past, a loss of a model of profesional development and also the loss of ensuring integrity in the work. Going back to a true apprenticeship model is part of the long, long way back for the Big 4.
So it isn't the fault of AI. It's the lack of the model. Wisdom and judgement b the way for Aristotle are part of the cardinal virtues.