Human control is more than a person at a screen
What does it mean to keep a decision under human control? The answer requires more than placing someone in front of a dashboard and asking for confirmation. If that person cannot understand the relevant criterion, request further information or halt the process, supervision may amount to little more than a formal gesture.
For anyone responsible for an organisation, the central question begins here: what authority remains with the person who must answer for the consequences? A human presence has little value if the surrounding process removes the time, information or permission needed to disagree with an automated recommendation.
Automation should follow the risk of being wrong
The underlying issue does not require a choice between accepting every form of automation and rejecting the use of artificial intelligence altogether. A more useful distinction is between tasks where an error can be corrected easily and decisions that may affect employment, income, access to services or the continuation of a commercial relationship.
The degree of autonomy granted to a system should reflect how difficult it would be to repair a mistake. Technical convenience is not an adequate measure of what may safely be delegated. A process can be simple to automate while still carrying consequences that demand careful human judgement.
This distinction brings the discussion into everyday management. Before asking which tool is suitable, a responsible organisation should decide what may be delegated, under which conditions and with what route of appeal. What would it mean to hold a manager accountable for a recommendation if that manager has neither the time nor the information required to challenge it?
A simple ranking can become a serious decision
Consider a hypothetical company using a system to rank supply requests. The tool recommends giving priority to customers whose previous records appear to suggest a lower risk of non-payment. Administratively, the proposal may look straightforward. Its significance changes, however, if the ranking determines access to a good that is essential to the customer’s own activities.
Now suppose a recently acquired customer appears at the bottom of the list because its history is short. Has the absence of information been treated as a sign of risk? Can the commercial manager distinguish that assumption from evidence of an actual payment problem? If not, accepting the recommendation may turn uncertainty into a practical disadvantage without anyone explicitly deciding that this should happen.
The point is not that the system must never use previous records. The point is that its output should not be treated as self-explanatory. A lack of evidence may be interpreted as caution, but it may also reflect a blind spot in the data. Someone must be able to recognise that distinction before the ranking determines what happens next.
Oversight needs practical powers
In this example, meaningful human control would require access to the relevant information, the authority to change the priority and a procedure for recording the reason for doing so. It would also require disagreement to be possible without being treated automatically as an unacceptable delay.
Asking staff to review recommendations carefully while judging them solely by the speed with which they execute those recommendations creates a conflict that the expression human supervision cannot resolve. An organisation cannot demand thoughtful scrutiny and reward only rapid compliance.
The people expected to intervene must therefore have the training, time and institutional support to do so. A review process that exists on paper but cannot be used during a busy working day is not a safeguard. It is a description of an intention.
The real cost begins after the recommendation
The economic analysis of an automated decision should include what happens after the recommendation has been issued. A quicker task may create additional work in verification, complaints handling, explanation or repair. Assessing delegation properly means considering the whole journey rather than the instant at which a list is produced.
In the hypothetical case, an unsuitable ranking might lead the customer to seek an explanation, the service team to open an incident and senior management to reconsider the supply decision. It would be misleading to judge the system only by the speed with which it arranged the requests. The management decision should also cover the quality of the resolution and the chance to correct the priority before harm occurs.
This broader view can change the apparent value of automation. Work removed from one stage may simply be transferred to another, perhaps to people with less authority and fewer resources. The relevant question is not how much effort the organisation avoids at the point of recommendation, but how much effort it creates elsewhere, within or beyond its own boundaries.
Who carries the burden of uncertainty?
There is also a decision about who bears uncertainty. If challenging an automated outcome requires a complicated route, much of the burden of control will fall on the person affected by the decision. That person may have to explain the problem repeatedly, provide information already held by the organisation or persist until someone agrees to investigate.
A prudent internal rule would make the intensity of review proportionate to the possible consequence. The more difficult it is to recover from a decision, the more demanding the verification should be before that decision takes effect.
This principle also applies to access arrangements and customer relationships. A person should not have to possess unusual persistence, technical knowledge or confidence to obtain a fair review. If the organisation knows that its process can produce uncertainty, it should design a route through that uncertainty rather than leave the solution to the individual who has been disadvantaged.
Internal limits matter under pressure
The debate about external rules and internal commitments can be translated into a practical management question: what prevents an organisation from relaxing its own limits when pressure builds to move faster? A policy that permits every exception for the sake of convenience offers little resistance when targets become demanding.
It is therefore sensible to decide in advance who may authorise a departure from the ordinary process, on what grounds and subject to what later review. An exception should not become an informal shortcut available to whoever happens to be closest to the workflow.
Clear limits also protect managers and staff. When the basis for a deviation is recorded, responsibility does not rest solely on the person who happened to approve it in a moment of urgency. The organisation can then examine whether the exception was justified, whether it should remain exceptional and whether the underlying process needs to change.
Responsibility must be visible
This discipline requires a clear distribution of responsibility. The person selecting the tool can assess whether it is suitable for the task. The person overseeing the workflow can identify practical difficulties in its use. The person accountable for the decision must have the authority to suspend the process.
These roles may be held by the same individual in a small organisation, but they should not remain implicit. When responsibilities are left undefined, each participant may assume that another person was responsible for checking the recommendation. After a failure, the absence of a clear owner becomes part of the failure itself.
Accountability is not secured by assigning blame after the event. It is strengthened when the process makes clear who can ask for more information, who can reject a recommendation and who must ensure that an affected person receives a meaningful explanation.
Exceptions need a route, not just a principle
For managers of services across a territory, the same reasoning can be applied to access criteria and referrals. In a hypothetical service, automating the initial sorting of requests would require a clear way to handle situations that do not fit the available categories.
An exception should have a defined route, so that a person is not dependent on insisting until they happen to find someone willing to listen. Categories are useful for organising work, but they should not become barriers that conceal cases the system was not designed to recognise.
The responsible question is not whether every unusual situation can be predicted in advance. It is whether the organisation has the capacity to notice when a case falls outside the pattern, pause the ordinary route and involve someone with the authority to decide what should happen next.
Ask what can be understood, interrupted and repaired
Before authorising a particular use, management can require a clear account of the task, the limits of the system and the review procedure. That account should make it possible to understand when a recommendation may proceed, when it must stop and who assumes responsibility for the next decision.
If those answers remain vague, the sensible response is to restrict the system’s autonomy until the process is clearer. Uncertainty about responsibility should not be treated as a reason to let the system continue without challenge. It is a reason to reduce the consequences of its recommendations.
The practical test can be simple: delegate only as far as the organisation retains a genuine ability to understand, interrupt and repair. That requires time for review, justified exceptions and evidence that corrections reach the people who need them. In your organisation or in the service you manage, who would really be able to stop an automated decision before it produces consequences?