SENTINEL
An exception-management workspace for capital-markets operations — AI triages, explains and learns; a human signs every decision. UX architecture, system logic and trust, designed as one system.
The humans who fix broken trades deserved a system that fights for them
After every market trade, a settlement pipeline runs — and the trades that break out of it land on an operations analyst’s desk with a ticking clock. I designed Sentinel: a workspace where AI triages, explains and gradually automates those breaks — while a regulated human stays accountable for every action. This case study goes deep on the three things that made it work: the UX architecture, the system logic, and the AI workflows — in the product and in my own process.
Hundreds of broken trades. One flat grid. Zero help.
Operations teams worked exceptions out of a paginated table built for storage, not for work — prioritisation lived in analysts’ heads, diagnosis took four modules, and fixes lived in tribal knowledge. Past the settlement deadline, every open break turns into penalties and fails.
MissionTurn a flat queue of failures into a prioritised, explained and increasingly automated flow of decisions — without ever taking the final call away from the human who signs the audit trail.
A trade isn’t done when it’s traded
Every executed trade still has to be matched, enriched, confirmed and settled — usually inside one or two days. Most flow straight through. The ones that don’t are exceptions — and they are the product’s entire reason to exist.
Failed validation, unmatched details, missing data, breached limits. Each one is money in limbo with a deadline — after it, the break becomes a settlement fail with real penalties. A team of 10–20 analysts can face thousands of these on a heavy day.
Four contacts on the scope
The legacy exception screen was a flat, thousand-row grid. Headcount scaled linearly with trade volume because the queue could only move as fast as humans could read it.
Prioritisation is blind
The grid sorts by age and amount — but urgency is really deadline × value × penalty regime × counterparty. The system had all four signals. It never computed them. Analysts kept the math in their heads, or in private spreadsheets.
Diagnosis is archaeology
One “SSI mismatch” meant visiting the trade, the counterparty’s instructions, the account master and the message log — four modules — before the analyst even knew what kind of problem they had.
Fixes live in people, not the system
Most breaks were recurring patterns with known resolutions — knowledge that took new analysts months to absorb and left with every resignation. Nothing captured it.
The system never learns
The same counterparty could cause the same break every Tuesday, and the platform treated each one as brand new. Zero memory, zero improvement — forever.
How might we turn a flat queue of failures into a prioritised, explained and increasingly automated flow of decisions — while keeping a regulated human accountable for every action? That last clause is load-bearing. It’s what makes this an interaction-design problem, not just an ML problem.
I didn’t ask what analysts do. I watched.
Shadowed analysts processing live queues — watching what they actually do, not what they say they do.
With a BA, mined which exception types occur most, how long each takes, and which modules analysts visit mid-resolution — the data that later set the Workbench’s “one click away” threshold.
Ops managers, risk officers and implementation consultants — the people who own the consequences.
The personal Excel trackers, printed cheat-sheets of counterparty quirks, and “anyone seen this break before?” chat threads — every one of them a requirement the legacy system had failed.
Analysts hunt in batches, not rows
They look for “all the breaks caused by this counterparty’s bad SSI today” and resolve them together. Their mental model is cause-first. The legacy UI’s one-row-at-a-time model fought them all day — this single insight drove the biggest structural decision in the product.
Nobody was anti-automation. They were anti-unexplained.
When probed on trust, analysts didn’t reject the machine making suggestions — they rejected not being able to see why. Explainability wasn’t a nice-to-have; it was the price of admission.
“I’ll click approve all day if I can see why. I’m the one who signs the audit trail, not the robot.”OPERATIONS ANALYST · CLIENT WORKSHOP
Pre-flight checks for every decision
No AI action without a visible, plain-language rationale. In a regulated workflow, explainability is the price of admission — not a feature.
AI compresses the investigation — what broke, why, what usually fixes it. The decision stays human until a pattern has earned autonomy.
Get the object model and the state machine right first. The UI is a projection of them.
The system earns automation rights per pattern, with human-controlled thresholds. There is no global “AI mode: on”.
The design target is the 3,000-exception morning after a market event — not the quiet Tuesday.
The object model came before any screen
In enterprise systems, navigation problems are almost always object-model problems in disguise. The legacy “archaeology” existed because the UI was organised by module while the analyst’s task cut across modules. So I mapped the domain as objects first — and gave it two objects it never had.
The product insight: the AI layer didn’t need new screens — it needed new objects. Once Root Cause (matching the analysts’ cause-first mental model) and Playbook (tribal knowledge turned into a system object) existed, every AI capability — triage, explanation, suggestion, automation — became an attribute of something users could already see, inspect and audit.
From the database’s org chart to the analyst’s day
The legacy IA mirrored the database. The new IA mirrors the three modes of an operations shift: monitor, resolve, govern.
Every diagnosis forced the analyst to carry context across four of these by hand.
The prioritised queue — exceptions grouped by probable root cause and ranked by computed urgency. One card says “SSI mismatch · Counterparty X · 23 trades”, not 23 rows.
One exception group, fully assembled: trade, counterparty, SSI, messages, history, AI diagnosis and suggested playbook in a single composed view. The analyst stops foraging.
Playbook library, automation thresholds, AI performance stats and the audit trail — governance as a user-facing surface, not an admin back room.
Defaults encode the new model; escape hatches respect the old one. Root-cause grouping is the default lens, and a toggle restores the flat trade-level view for the analysts and auditors who need it. The Workbench rule was measurable: every data point needed for 90% of resolutions must be visible or one click away — validated against the workflow-mining data, not taste.
Navigation rules that survive the worst day
Computed, not inferred — and inspectable
The queue ranks by deadline × value × penalty × track record. Hovering the urgency chip shows the factors. The explainability principle applied before any ML enters the picture.
Two levels deep. Never three.
Radar → Workbench → detail drawer. Anything deeper surfaces as an overlay, so the analyst never loses queue context mid-resolution.
Designed at 3,000. Demoed at 50.
Virtualised groups, a persistent “next most urgent” action for flow-state processing, and bulk actions as first-class — root-cause groups made “resolve all 23” a safe, reviewable operation.
The exception lifecycle is a designed state machine
Defined with engineering before high-fidelity UI — because every state needed an answer to: who can act, what can they do, what does the AI do, and what gets audited.
A visible state, not a background process — auto-resolutions stream in a side rail, each one pausable mid-flight. On failure or timeout it degrades gracefully into the human queue, carrying its full attempt history.
Ranked by computed urgency, grouped by root cause. Parking an exception requires a reason and a timer — parked breaks re-surface automatically and can never silently rot.
- Auto-resolution needs three keys, not one: confidence above threshold θ, a playbook a named human explicitly granted automation rights, and trade value under the auto cap. Any key missing — human queue.
- Deadlines outrank everything: if a market cut-off passes while an exception is parked, it auto-escalates to breach-risk, urgency recomputes, the manager is notified. Parking never suppresses deadline logic.
Who holds which dials — and what happens at the edges
| Capability | Analyst | Senior analyst | Ops manager | Auditor |
|---|---|---|---|---|
| Resolve exceptions | ✓ | ✓ | ✓ | — |
| Approve four-eyes verifications | — | ✓ | ✓ | — |
| Create / edit playbooks | propose | ✓ | ✓ | — |
| Grant playbook automation rights | — | — | ✓ | — |
| Adjust confidence thresholds (θ) | — | — | ✓ | — |
| View full audit trail | own actions | team | team | ✓ all |
The separation that matters: the people who benefit from automation (analysts) are not the people who authorise it (managers). Deliberate — it mirrors how financial-controls governance already works, which made the AI layer legible to compliance teams on day one.
Two analysts open the same group
Soft lock with presence — “Priya is working on this” — plus a read-only shadow view. Not a hard error.
AI confidence flips mid-resolution
Non-blocking banner: “Diagnosis updated — review changes” with a diff of what changed. The analyst’s context is never silently replaced.
Counterparty correction arrives during auto-resolution
Auto-resolution halts, the exception re-triages, and both events are audit-logged with a causal link between them.
Cut-off passes while parked
State auto-escalates to breach-risk, urgency recomputes, the manager is notified. Parking never suppresses deadline logic.
No playbook match, low confidence
An honest empty state: “No known pattern — here’s the assembled evidence.” The system shows its homework even when it has no answer.
A playbook’s success rate drops below floor
Automation rights auto-suspend, the manager is notified, and the playbook returns to suggest-only mode. Trust that can’t be lost isn’t trust.
The trust ladder: how a pattern earns the right to act
The core logic governing the human–AI relationship. Every playbook climbs four rungs — and can be demoted. This is a design artifact, not an ML artifact: it exists so a manager can stand in front of a regulator and narrate exactly why the system was allowed to do what it did. The interaction design is the compliance story.
Shadow
AI predicts the resolution silently. Prediction logged, never shown. Builds the accuracy track record with zero user exposure.
↑ promoted when accuracy holds over N real casesSuggest
Diagnosis and suggested playbook appear in the Workbench. The analyst applies it manually. Every accept and reject is logged.
↑ promoted when acceptance holds over M suggestionsOne-click
The resolution arrives pre-filled; the analyst reviews and confirms in a single action. Still human-executed, every time.
↑ promoted only by a named manager’s explicit grantAuto
Executes without a human — under the value cap, visible in the auto-resolution rail, pausable mid-flight, fully audited.
the only rung where the machine acts aloneDemotion is automatic at every rung. If a playbook’s success rate decays below its floor, it drops back down the ladder and the manager is notified — autonomy is rented on evidence, never owned.
Every explanation speaks the same grammar
Analysts scan hundreds of diagnoses a day. A fixed What / Why / Fix structure becomes pre-attentive after one shift — free-form AI prose would be slower to parse, harder to audit, and dangerously convincing when wrong.
SSI mismatch — beneficiary account differs from the counterparty’s instruction.
Fields BENEF_ACCT differ between the trade and the incoming message; this counterparty updated SSIs two days ago; 14 similar breaks today share this signature.
Playbook “Refresh SSI from custodian feed” — the strongest match across similar historical cases.
One-click eligible. Pre-filled resolution offered for single-action confirm.
Suggest only. Diagnosis shown with evidence; the analyst drives.
Evidence-only mode. No recommendation — just the assembled homework.
Confidence is worded, not scored. “87% confident” invited false-precision debates in testing; three bands with defined behavioural consequences did not. Raw numeric scores still live in the Control Room, where managers tune θ.
One-click “reject diagnosis” unlocks manual mode and captures why — feeding the retraining queue.
A visible freshness indicator on every diagnosis — the system admits when its world has moved.
The honest empty state. An AI feature without designed failure states is a demo, not a product.
I designed the same deal for myself that I designed for the analysts
AI made me faster at everything except judgment — which is precisely the deal the product offers its users.
Synthesis at scale
Workshop transcripts, tickets and audit-log extracts went through LLM-assisted clustering to surface candidate exception patterns — it found a recurring long-tail break I’d have needed weeks to spot manually.
RULE — AI proposes clusters; I verify every one against source quotes before it enters the research wall.Synthetic worst days
Generated schema-constrained synthetic datasets — thousands of exceptions with plausible types, values and deadlines, reviewed by a BA — so usability tests ran on the 3,000-row morning, not a 12-row happy path.
RULE — lorem-ipsum data would have invalidated the core design claims.Adversarial flow passes
Before each design review I ran the state machine and edge-case docs through an adversarial AI pass — “enumerate what this logic doesn’t handle” — then triaged the output with engineering. The edge-case matrix roughly doubled this way.
RULE — the AI attacks the logic; it never writes it.Disposable coded prototypes
Queue flow, the pausable auto-rail and the diagnosis diff banner were AI-assisted coded prototypes, not click-throughs — flow-state processing needs real keys and real data density to test honestly.
RULE — the prototypes were throwaway; the validated patterns were the deliverable.Four bearings, plotted and argued
A smarter flat list — familiar, cheap to build, and exactly what every competitor ships.
Root-cause grouping as the default lens — the UI finally matches the analysts’ cause-first mental model, and bulk resolution becomes safe.
AI as a chat assistant — “ask me about this exception”. Flashy in demos.
AI woven into the objects — diagnosis on the exception, suggestions on the group, stats on the playbook. Structured beats conversational in regulated workflows.
A global automation toggle per client — the fastest path to a headline automation number.
The per-playbook trust ladder — automation grows pattern by pattern, on evidence, with a named human granting each promotion.
Numeric confidence scores everywhere — engineering’s instinct, and honest on paper.
Worded bands for analysts, numbers for managers — each band with a defined behavioural consequence instead of a precision debate.
Three simulation runs, three course corrections
Do cause-groups make sense?
- Signal
- Analysts predicted group contents correctly — but refused to trust bulk actions without seeing the group’s own confidence.
- Correction
- Added the per-group confidence chip and a “show outliers” affordance.
Does flow mode actually flow?
- Signal
- “Next most urgent” beat browsing on throughput — but analysts felt a loss of control and pushed back.
- Correction
- Flow mode became opt-in per session. Efficiency that removes perceived agency gets rejected by expert users even when it’s faster.
Fixed grammar vs free prose?
- Signal
- Fixed What/Why/Fix cut time-to-first-action — and analysts correctly rejected bad diagnoses more often. Prose was too convincing when wrong.
- Correction
- The grammar became a hard product rule, not a style preference.
End-of-watch debrief
Stated honestly: these are the design targets the system was engineered against — with one outcome that is architectural, and true by design.
Every automated step runs under the trust ladder, under a value cap, visibly, with a named human having granted the rights. Not a metric — a property of the system.
The four-module archaeology collapsed into one assembled Workbench view — the design claim the workflow-mining data was recruited to defend.
Recurring patterns resolved through captured knowledge instead of tribal memory — climbing the ladder pattern by pattern.
Playbooks as institutional memory: the knowledge that used to leave with every resignation now lives in a system object.
Three things I’m keeping
Screens were the cheap part. Introducing Root Cause and Playbook as first-class objects is what made every AI capability explainable, governable and navigable.
The trust ladder turned “do users trust the AI?” from a research anxiety into an engineered, observable, adjustable mechanism.
The empty state that admits “no known pattern”, the banded confidence, the visible demotions — every honest moment bought more adoption than any clever one.
“AI made me faster at everything except judgment — and designing that deal twice, for the analysts and for myself, was the most honest work in the project.”
END OF TRANSMISSIONFour surfaces carry the whole system
A walk through the four surfaces — use the arrows to move between screens.
