Hybrid intelligence

Gareth McConomy

AI researcher

My research asks when people and AI make better decisions together, and what evidence justifies relying on the result.

Selected researchAbout me
Gareth McConomy

Research interests

Evaluating and assuring
hybrid intelligence.

Technical assurance means establishing how a workflow performs, where it can fail and what human oversight contributes. I study these questions in analytical work that depends on evidence, interpretation and professional judgment.

Decision quality

When does a combined workflow improve on expert practice or AI alone? How should accuracy, uncertainty and the effort needed to reach a sound judgment be compared?

Human oversight

What enables experts to detect errors, challenge assumptions and make effective use of AI? How do the timing of assistance and the design of review affect the result?

Evidence for reliance

What evidence supports using a workflow for a particular purpose? How should its limits be established and reassessed as models, tasks and working arrangements change?

Selected research

Methods,
evidence & limits.

These studies test reasoning methods, calibration and evidence use in AI risk assessment. They provide evidence about particular parts of a workflow, while raising further questions about how experts use and review the results.

HHAI 2026Human–AI assurance

Layer-Appropriate Augmentation for Human-AI Risk Assessment: Integrating Case-Based Reasoning with Fuzzy Calibration

Testing how reasoning methods and output calibration work together, with performance assessed against an expert reference.

About the study

Across 2,880 trials, this study examines reasoning scaffolds and output calibration together. Benefits depend on the model, task and chosen expert reference; the design principles are to structure reasoning, calibrate to context and validate the combination.

Gareth McConomy, Raymond Bond, David Glass, Jun Liu and Holly Toner

Read paperPDF
FLINS–ISKE 2026Calibrated confidence

Bridging Causal Reasoning and Fuzzy Calibration for Trustworthy AI Risk Assessment

Examining when calibration improves agreement with an expert reference and when an intervention can make reasoning worse.

About the study

The study tests causal risk patterns and Ordered Weighted Averaging across 125 case-pattern pairs. Calibration benefits are specific to the chosen reference and context; imposing fuzzy structure within reasoning produced different, adverse effects.

Gareth McConomy, Jun Liu, David Glass, Raymond Bond and Holly Toner

Read paperPDF
ICCBR 2026 workshopEvidence & traceability

Can LLMs Reason with Case-Based Structure? Model-Dependent Effects of CBR-Inspired Decomposition on AI Risk Assessment

Testing whether a reasoning scaffold helps across models, and examining the limits of transferring a successful method.

About the study

A 360-run study compares unstructured access, chain-of-thought, case-based decomposition and multi-agent deliberation. The central finding is that a scaffold that improves one model may fail to help another, supporting validation for each model and task.

Gareth McConomy, Raymond Bond, David Glass, Jun Liu and Holly Toner

Read paperPDF
ICAD 2026 · in pressCausal risk assessment

Augmenting Large Language Models with Causal Risk Patterns for AI Deployment Risk Assessment

Evaluating whether structured causal knowledge helps an AI assessment connect possible harms to the evidence in a case.

About the study

The research evaluates an 87-pattern causal knowledge base across five real-estate AI deployment cases. It examines how structured risk knowledge affects causal depth and evidence grounding, with professional review remaining part of evaluation.

Gareth McConomy, Raymond Bond, David Glass, Jun Liu and Holly Toner

Read paperPDF

Essays & perspectives

Questions for
research and practice.

About me

Research informed by
professional practice.

I am a doctoral researcher at Ulster University, working on hybrid intelligence and the assurance of human–AI workflows.

Before moving into research, I spent more than two decades in real estate tax and finance, including roles in Big Four practice and international real estate firms. I also founded and ran my own advisory firm. That experience informs my research into the quality of expert judgment, the work required to support it and the conditions under which AI adds value.

Education & research

I hold an MSc in Computer Science and Data Analytics from the University of York, an MBA from Ulster Business School, and a postgraduate diploma in Financial Strategy (with distinction) from Saïd Business School, University of Oxford.

I have authored several peer-reviewed papers in artificial intelligence and presented my research at AI conferences in the United States and Europe.

Contact

Correspondence
& enquiries.

For research discussions, collaboration or speaking invitations, please email me.

Research profile on ORCID LinkedIn

Human–AI collaboration

The unit of evaluation is the workflow

A good model assessment is only one part of a sound decision process. The handoffs, review and human contribution need examining too.

By Gareth McConomy5 min read

A model can produce a sound assessment and still be part of a poor decision process. Its answer may arrive after the important judgment has already been made. A reviewer may see the conclusion without the evidence needed to challenge it. An uncertainty that was visible during analysis may disappear when the result is summarised.

These are questions about the arrangement of the work. They are central to hybrid intelligence because the behaviour of the combined process depends on what each participant contributes, what each can inspect and how one contribution changes the next.

For analytical work, I think the useful unit of evaluation is the workflow that produces the judgment. A model evaluation is an essential part of that inquiry. It leaves open what happens when the output becomes an input to someone else's reasoning.

Follow the judgment through the process

Consider a team reviewing a proposal. An AI extracts relevant evidence, identifies possible difficulties and drafts an assessment. An expert checks the draft before making a recommendation. This description sounds straightforward, but it leaves several consequential choices unspecified.

Does the expert form an initial view before seeing the AI's assessment? Can they inspect the source material directly? Are omitted issues visible anywhere? What happens when the expert disputes the model's interpretation? Does the revised assessment preserve the reason for the change, or simply replace the earlier answer?

Each choice creates a different process. Showing a draft first might save effort, while also making that draft the starting point for the review. Requiring an independent assessment may provide a useful comparison, but consume much of the time the system was intended to save. These are trade-offs to investigate in the intended setting.

The handoff deserves particular attention. It is the point at which one participant's uncertainty, assumptions and evidence must become intelligible to another. A concise conclusion may be convenient to read while being difficult to examine. A long explanation may reveal the structure of an argument while imposing too much review effort. The evaluation needs to consider both.

Specify the human contribution

The phrase “human oversight” can conceal very different activities. A person might check whether cited evidence exists, judge whether a causal mechanism fits the case, choose the organisation's tolerance for uncertainty, or authorise an action. These activities call for different information and different expertise.

A review stage should therefore have an explicit purpose. If it is intended to catch unsupported factual claims, reviewers need access to the relevant sources. If it is intended to challenge the framing of a problem, a checklist of extracted facts will be insufficient. If the reviewer controls a decision threshold, they need to understand what the score represents and how it behaves.

This also makes the limits of review easier to investigate. A reviewer may correct a conspicuous error while missing a material omission. Counting edits alone would tell us little about that difference. Useful evidence would include the kinds of errors that reach the reviewer, those they identify, those they introduce and those that remain in the final assessment.

Compare processes on the same question

An evaluation can begin with a small set of clearly specified alternatives: the current expert process, an AI assessment under defined conditions, and a proposed combined workflow. The point of comparison is the same underlying analytical task, with equivalent access to the information available at the time.

Decision quality requires more than a single accuracy figure. For document-based analysis, some claims can be checked directly against evidence. Other conclusions depend on interpretation or a stated decision policy. Those should be assessed separately, so that disagreement about judgment does not hide a factual mistake, and factual consistency is not mistaken for a sound overall conclusion.

Effort belongs in the comparison too. A workflow that improves the first draft but doubles the time spent verifying it may have a different value from the one initially expected. Conversely, a slower process may be justified if it reliably exposes a consequential issue that was previously missed. The relevant trade-off depends on the task and the cost of its errors.

I would also examine how performance varies between cases. An average can obscure whether the workflow helps with routine assessments while failing on the ambiguous cases for which expert involvement matters most.

What my research contributes

In our study of layer-appropriate augmentation, we examined how structured reasoning and output calibration interact in an LLM-assisted assessment pipeline. The findings were conditional on the model, task and expert reference. They support treating the arrangement of the method as something to evaluate explicitly.

That study does not establish how every aspect of a live expert–AI interaction will behave. It evaluates assessments against an expert reference; it does not measure the effects of repeated use on a working team's attention, review habits or decisions. Those are further empirical questions. The distinction matters when moving from a promising analytical method to an operational workflow.

My interest is in making that transition inspectable. We should be able to explain what the AI contributes, what the expert contributes, where either can correct the other and what evidence supports the resulting judgment. Those explanations provide a basis for testing the process again when its model, task or human review arrangements change.

Research in practice

When faster analysis becomes better work

The value of AI assistance depends on what happens beyond the first output: review, judgment and the decisions the work supports.

By Gareth McConomy5 min read

An AI system may shorten an analytical task without changing the economics of the work around it. A faster draft still needs review. Time released from one activity may be absorbed by another. An organisation may produce more analysis without becoming any better at deciding what to do.

A useful starting question is what becomes more valuable if the proposed approach works. The answer could be greater capacity, a better service, a decision reached sooner or a materially better understanding of the alternatives. Each possibility has a different route from technical capability to commercial benefit.

That relationship deserves as much attention as the performance of the model. It connects the evaluation of a technical method with its use in practice.

Start with the work that matters

A useful starting point is a specific piece of work and the person who relies on it. What decision does it support? What makes the current process expensive, slow or difficult? Where does expertise make a difference to the outcome?

Consider a hypothetical advisory team preparing an assessment. AI might help retrieve evidence, compare scenarios or draft the explanation. Those activities serve different purposes. Faster retrieval could release expert time for interpretation. Wider scenario coverage could reveal an alternative that deserves attention. A clearer explanation could make the advice easier for the client to use.

The commercial question changes with the proposed contribution. If the objective is capacity, the evaluation needs to establish what additional work can actually be completed. If the objective is better advice, it needs to identify what becomes more useful or dependable. A reduction in drafting time alone would establish neither.

Trace the benefit through the whole process

The attractive number in a demonstration is often the time taken to produce an output. For an operational decision, I would examine the effort required to produce an acceptable result, including preparation, checking, correction and approval.

Suppose a task previously took four hours. AI reduces the initial work to one hour, but introduces ninety minutes of additional review. In this deliberately simplified example, the time released is ninety minutes. Whether that creates value depends on what happens next, including any change in quality and the cost of maintaining the approach.

If a senior reviewer is already the limiting resource, more drafts could increase the queue. If the released time is scattered across people who cannot use it for additional work, a nominal capacity gain may be difficult to realise. If it allows a team to respond before a decision deadline, the benefit could be greater than the labour saving suggests.

I would therefore distinguish time released, usable capacity and financial benefit. Moving from one to the next requires assumptions that should be made explicit.

Ask how the service could change

Efficiency is one possible source of value. AI may also make it feasible to offer a different service: examining more alternatives, updating an assessment more often or giving a client a clearer account of uncertainty.

These possibilities raise questions about demand and delivery. Would the additional analysis change a client's decision? Would the client value a continuing relationship over an occasional report? Who would review the updates, and what would trigger a fresh professional assessment?

Pricing belongs in that discussion. For work charged by time, fewer hours can affect revenue as well as cost. For a fixed-fee engagement, reduced effort may improve the economics, provided quality and scope remain controlled. A continuing service creates different expectations about availability and responsibility. None of these arrangements makes the investment attractive automatically.

My interest is in connecting the design of the analytical work to the promise made to the client. A technically feasible service still needs a credible reason for someone to buy it and a workable way to provide it.

Treat assurance as part of the value question

Confidence in the process affects what an organisation can responsibly do with it. If an output requires complete re-performance before it can be used, that review burden belongs in the assessment of value. If a well-designed method makes evidence and assumptions easier to examine, the review may become more useful.

The evaluation should reflect the intended use. An exploratory analysis can be helpful without supporting a final recommendation. An assessment used to commit resources needs evidence appropriate to that decision. The required human contribution, acceptable uncertainty and consequences of error should be considered together.

A workflow-level evaluation connects these questions to the way the work is organised. It does not supply a universal return on investment. Commercial value must be examined in the setting where the work will happen.

Make the investment testable

Before expanding an approach, I would want a clear account of the current process, the proposed change and the benefit expected. A bounded trial can then examine quality, total effort, review demands and whether the intended user finds the result more useful. It should include difficult cases as well as routine ones.

The result may support a different ambition from the original proposal. AI might be valuable for preparing evidence while remaining unsuitable for drafting conclusions. A method might justify a richer service without reducing its cost. Another may be technically impressive yet offer little advantage over current practice.

A sound decision connects those findings to the business: what becomes possible, who benefits, what it costs and what would justify continuing. Those questions connect technical performance with the usefulness of the work.

Reasoning methods

When structure changes the reasoning

Knowledge, reasoning procedures and output calibration are different interventions. Their contribution has to be tested at the right layer.

By Gareth McConomy5 min read

It is tempting to respond to an unreliable AI assessment by adding structure. Break the task into stages. Require an explanation for each step. Introduce another model to review the answer. Ask for more explicit reasoning.

Each intervention has a plausible rationale. Each also changes the task the model is being asked to perform. The important question is whether the particular change improves the result under the conditions that matter.

This has been a recurring theme in my research. “More structure” is too broad a description to tell us what an intervention will do. We need to identify the structure, the layer at which it operates and the outcome by which it will be judged.

Three different places to intervene

One intervention changes the knowledge available to the model. A causal risk pattern, for example, can describe the conditions under which a problem arises and the mechanism through which it develops. Supplying that pattern changes what the model can draw upon when examining a new case.

Another intervention changes the procedure. The model might be asked to match conditions to evidence, trace a mechanism and then assess its applicability. Here the instruction specifies how to organise the analysis, even if the underlying information remains unchanged.

A third intervention changes how outputs are combined or interpreted. Several assessments might be aggregated according to an explicit rule. This operates on the results of reasoning and can change the final score without improving any individual explanation.

These distinctions matter because an improvement at one layer does not establish that a similar-looking intervention will help at another. An aggregation rule can make a set of scores more consistent with an expert reference while leaving the original reasoning untouched. A detailed reasoning template can make an answer easier to inspect without making it more accurate.

Hold the information constant

In our study of case-based reasoning structure, the prompting conditions had access to the same causal risk knowledge. The comparison examined how different ways of organising the task affected performance: unstructured access, chain-of-thought, case-based decomposition and multi-agent deliberation.

The benefit of case-based decomposition differed between models. One improved substantially on the tested task; another did not show a significant improvement. A further model encountered an output-format incompatibility under the tested configuration. That last result should not be interpreted as proof that the model lacked the underlying reasoning capability.

Holding the knowledge constant helped isolate the contribution of the procedure. It also bounded the conclusion: this experiment did not test whether access to the knowledge itself was better than having none. Keeping that distinction visible is part of understanding what the evidence actually supports.

The approach was inspired by case-based reasoning. It was not a complete case-based reasoning system with formal similarity measures and automated case retention. Naming a method carefully matters because the name can otherwise imply capabilities that have not been implemented or evaluated.

Better explanations and better assessments

A structured response can be useful even when it does not improve the headline performance measure. It may expose which condition was judged present, which passage supported that judgment and where the argument depends on an interpretation. That can make review more targeted.

But inspectability is a separate claim. A citation can exist without supporting the conclusion attached to it. An orderly chain of statements can contain an invalid step. A causal mechanism can be plausible in general while lacking an enabling condition in the case being assessed.

I would therefore evaluate the explanation and the assessment separately. Does the explanation accurately represent the supplied evidence? Can a reviewer locate the disputed step? Does the final assessment improve against the chosen reference? Does the structure make consequential errors easier to find, and at what cost in review effort?

The causal-pattern augmentation study examines another part of this problem: whether explicit causal knowledge helps models move from naming risk categories to explaining how a risk could unfold. The results support further investigation of that approach, while the variation between models and the limits of the evaluation constrain its generality.

Complexity creates its own questions

Adding another agent introduces another interpretation of the task and another opportunity for the analysis to change. Agreement between agents may reflect useful corroboration, but it may also reflect shared evidence, similar assumptions or the influence of an earlier answer. The number of agents does not by itself establish independence.

In the case-based structure study, the multi-agent deliberation tested reduced performance. This is evidence about that configuration, rather than a verdict on every form of multi-agent reasoning. It is nevertheless a useful reminder that an elaborate architecture needs a demonstrated contribution from its additional stages.

A practical research approach is to begin with a defensible baseline, change a specified component and examine what changed in the output. Record the model version, instructions, available knowledge and evaluation criteria. Investigate failure cases as well as average improvements. Revisit the finding when a material component changes.

For me, the value of structure lies in what it makes possible: better use of evidence, more useful reasoning or more effective human scrutiny. Those are empirical questions. A carefully designed workflow should make them easier to answer.