An AI Boss Fired a Human—but It Wasn't Acting Alone
The real story is not whether the algorithm reached a reasonable conclusion. It is whether anyone can identify who observed, judged, authorized, and owned the decision.
An employee at Andon Market was reportedly late for 17 of 23 shifts. According to the company, he received repeated warnings and additional training, but the lateness continued.
If you were his manager, would you fire him?
The answer may feel obvious. But one fact changes the nature of the question: the recommendation came from an AI manager named Luna.
The resulting headline practically writes itself—an AI boss fired a human. Yet that compressed version hides the most important part of the story. Luna did not independently retrieve every relevant rule, make an employment decision, and carry it out. Humans prompted the system to recover an attendance policy it had not initially surfaced, reviewed its recommendation, and executed the termination.
Once the sequence is expanded, the central question is no longer simply whether the employee should have been fired.
It is: Who actually made the decision?
The Outcome Can Look Clear While the Process Remains Unclear
Andon Market is a real San Francisco retail experiment operated by Andon Labs. Luna, its AI manager, can select products, communicate with employees, monitor operations, and use digital business tools.
According to Business Insider's reporting, Luna recommended “parting ways” with the employee after the attendance problems continued. Human staff reviewed that recommendation and carried it out.
The recommendation may have been reasonable. A human manager looking at the same record might have reached the same conclusion.
That is precisely why the case matters.
When an outcome seems justified, people tend to stop examining the process that produced it. The quality of the result becomes a substitute for the legitimacy of the system. But a process that appears harmless in an easy case can become dangerous when the evidence is incomplete, the policy is ambiguous, or the consequences are difficult to reverse.
We do not have the employee's complete account, so this is not an attempt to retry the termination from a distance. The public record does not establish every relevant detail. What it does reveal is a decision system in which information, judgment, authority, and action were distributed between humans and software.
Calling that entire chain “the AI's decision” makes accountability harder to see.
A Decision Is a Relay, Not a Single Moment
Consequential decisions rarely happen in one step. They are relays in which responsibility changes hands several times.
The Andon Market case can be separated into seven stages:
- Observe: What happened?
- Retrieve: Which records, policies, and context matter?
- Judge: Does the evidence cross the threshold established by the policy?
- Recommend: What should happen next?
- Review: Is the recommendation supported, lawful, and fair?
- Authorize: Who has the power to approve the action?
- Execute: Who carries it out?
Afterward, a feedback stage should answer additional questions: Can the affected person appeal? Who investigates an error? What changes if the system fails?
In this case, store records and reports supplied observations. Humans prompted the retrieval of the attendance policy. Luna evaluated the situation and generated a recommendation. Humans reviewed the output and carried out the termination.
The public reporting, however, does not identify the individual who formally authorized the final employment action.
That gap matters. It does not mean Luna contributed nothing, and it does not prove that the process was improper. It means the phrase “the AI decided” is too vague to assign responsibility. The most dangerous handoff in a decision system is often the one nobody clearly owns.
“A Human Was in the Loop” Is Not Enough
Organizations frequently defend AI-assisted decisions by saying a human reviewed the output. That sounds reassuring, but human presence alone does not guarantee meaningful oversight.
A reviewer may see only the AI's conclusion rather than the underlying evidence. They may lack the time, expertise, or authority to challenge it. The interface may present the recommendation as the default. The organization may measure reviewers by speed or agreement, quietly punishing disagreement.
Under those conditions, the human is not exercising independent judgment. They are supplying a signature.
Meaningful review requires the ability to change the outcome. A reviewer should be able to inspect the evidence, request missing context, question the applicable standard, record a disagreement, and stop the action when necessary.
This distinction is consistent with the NIST AI Risk Management Framework, which emphasizes defined accountability structures and clearly differentiated human and AI roles. It also matters legally: the U.S. Equal Employment Opportunity Commission has made clear that existing employment discrimination protections continue to apply when organizations use algorithmic systems.
Delegating analysis to software does not delegate away institutional responsibility.
The Higher the Stakes, the Stronger the Review
Not every AI-assisted decision needs the same safeguards.
If an AI recommends a reversible product placement or proposes a routine inventory change, lightweight review may be enough. Errors are relatively easy to detect and undo.
But if a system recommends who loses a job, receives medical care, gets access to credit, has a transaction frozen, or is denied an essential service, the standard must be much higher. Those decisions affect income, health, opportunity, and legal rights. They can also be difficult to reverse after the harm occurs.
Before an AI output changes someone's life, an organization should be able to answer five questions:
- What may the AI observe? Define the data it can access and the limits of that data.
- What may it recommend? Separate analysis from authority to act.
- Who reviews it, and what evidence must they see? A conclusion without supporting context is not enough.
- Who can authorize and execute the action? Name the accountable role rather than referring vaguely to “the business.”
- How can the affected person challenge the decision? Appeals are part of the decision system, not an optional extra.
Organizations should also measure whether review is real. Useful signals include override rates, requests for additional information, review time, sampled accuracy, successful appeals, reversals, and disparities between groups.
A very high agreement rate should not automatically be celebrated. It may indicate excellent recommendations—or reviewers who never had a practical opportunity to disagree.
Who Evaluates the Evaluator?
The Andon Market case contains one final twist.
Luna's assignment was to operate the store and make it profitable. Andon Labs publicly lists memory and decision-making among the system's weaknesses, and the store had not yet become profitable when the termination story was reported.
That does not prove Luna's employment recommendation was wrong. Performance in one domain does not automatically invalidate a judgment in another. But it raises a more important governance question: Was the system validated for the role it was being asked to perform?
Before allowing an AI to evaluate a worker against a standard, someone must evaluate the AI against a standard too.
That means testing whether the system retrieves the right policies, handles conflicting information, recognizes missing context, communicates uncertainty, and behaves consistently across comparable cases. It also means ensuring that the human reviewer has both the competence and the organizational power to challenge the system.
The evaluator needs standards. So does the person reviewing the evaluator.
The Real Decision Was Distributed
So, who fired the employee?
Luna generated the recommendation. Humans at Andon Labs retained the organizational power to accept it and end the employment relationship. The process was distributed, authority remained human, and individual accountability is unresolved in the available reporting.
That answer is less dramatic than “an AI fired a human,” but it is more useful.
AI systems increasingly participate in decisions that shape jobs, money, care, and access. The key question is not merely whether a machine was involved. It is whether each stage of the decision has a defined owner, whether the reviewer can genuinely intervene, and whether the person affected has a path to challenge the result.
The AI evaluated the employee. The harder—and more important—question is who evaluated the AI.
Comments
Post a Comment