# Control and oversee the Investigator

Source: https://help.didit.me/investigations/investigator-automation-and-oversight
Updated: 2026-10-23

Lower the Investigator's autonomy by country, workflow or amount, add locks it never crosses, and oversee it with consistency checks and drift protection.

Open **Investigator → Settings** to set how much the Investigator may decide in
each queue, lower it for the items that match a segment, and add locks such as
"never auto-approve when the face match is below 70%". Then watch it in
**Performance** and **Activity**: consistency over re-runs, drift, service
levels, the QA of your own reviewers and a validation report you can hand to an
auditor.

> **Note:** These controls are rolling out one by one and may not be switched on for your
> application yet. Segments and locks can be saved before they are switched on,
> but they only take effect once they are. Changing them needs permission to
> change the application's settings, and every change is recorded in
> **Activity**.

## Go back to shadow mode, turn it off or run the setup again

Your application is always in one of three stages:

| Stage | What happens |
|---|---|
| **Not set up** | The Investigator reviews nothing: items reach your reviewers without a report |
| **Shadow mode** | It writes a report on every item and scores itself against your team's decisions, but decides nothing. Reviews in shadow mode are never billed |
| **Live** | It decides on its own where a queue's level allows it, and finished reviews are billed |

- **Back to shadow mode.** Ask Didit Copilot to put the Investigator back in shadow mode, or change the stage through the API. It keeps writing reports and scoring itself, but decides nothing until you move a queue to **Auto** or **Full auto** again in **Settings**. To stop automated decisions only for a while, use **Pause automation** instead: every item goes to your team, with its report.
- **Turn it off.** Ask Didit Copilot to turn the Investigator off, or change the stage through the API. It stops reviewing, and items reach your reviewers without a report, as before the setup. Your policy, your SOP and your settings are kept. To turn it on again, press **Automate reviews** and finish the setup: it starts in shadow mode.
- **Run the setup again.** **Run the setup again**, in the Investigator's menu, opens the setup with your saved answers. Finishing it saves your answers to your policy and starts the rollout over: every queue you pick goes back to **Suggest** and scores itself against your team for the length you choose, and a queue you leave out stops deciding on its own. A queue set to **Off** that you leave out stays as it is.

Each stage change is written to **Activity**. Changing the stage or running the setup needs permission to change the application's settings.

## How much it may do on its own

Each queue has its own level:

| Level | What happens |
|---|---|
| **Suggest** | The default. It writes reports; people decide every item |
| **Auto** | It decides items at or above your confidence threshold; the rest go to people |
| **Full auto** | It decides everything your policy covers, and a sample goes to a person for quality checks |
| **Off** | Items reach your reviewers without a report |

Moving a queue to **Auto** or **Full auto** needs a backtest and your explicit acknowledgement. **Pause automation** sends every item back to your team at once. You can also set a lower level for some countries, workflows or amounts, add your own locks, and let drift protection step a queue down: see [Control and oversee the Investigator](https://help.didit.me/investigations/investigator-automation-and-oversight).

## When it may decide on its own

On **Auto** and **Full auto**, the Investigator decides an item only when every condition in **Settings → When it may decide on its own** holds. Everything else stays with a person, with the report ready.

| Condition | What it means |
|---|---|
| **The item is covered by your policy** | Always. Anything your policy doesn't cover comes back to your team as a question |
| **Confidence at or above the queue's threshold** | Always. The threshold is 90% unless you changed it |
| **No open questions on the item** | Always |
| **No possible sanctions match on the item or anything linked to it** | Always. Those go to a person |
| **Transaction amount up to** | 5,000, 10,000, 25,000 (the default), 50,000 or 100,000 in your application's default currency, or **No limit**. Larger transfers always go to a person |
| **Most automated decisions per queue per day** | 50, 100, 200 (the default), 500 or 1,000, for every queue, per UTC day. Beyond it, items wait for a person with the report ready |
| **Send 5% of Full auto decisions to a person for QA** | On or off, for every queue. It keeps measuring accuracy after launch. The share is the QA sample you chose in the setup (2%, 5% or 10%), and switching it on here sets 5% |

Each change is written to **Activity**.

## Override a decision it took on its own

A verification or a case the Investigator decided on its own shows **Decided by the Investigator** on its record, with the action, the evidence and the confidence. Press **Override** (on a case, **Override with a reason**), then pick another action, why the decision doesn't fit and your reason. Your decision is applied to the record the way any decision a person takes is, and is saved to **Events** with your name and your reason, and to **Activity** under **Overrides**. The Investigator's own decision stays on the record next to yours. It was billed when it was taken; the override is never billed. You can override it again later.

You can override only while the Investigator's decision, or the last override of it, still stands. Once someone changed the record's status since, on its page, in bulk or through the API, or a new review decided it again, change it from the record itself.

## Autonomy by segment

Each queue has a level (**Off**, **Suggest**, **Auto** or **Full auto**) and a confidence threshold. **Autonomy by segment** lowers that level, or raises the threshold, for the items that match. Each segment has a name, conditions that must all match, and what happens then:

| Condition | Matches when |
|---|---|
| **Countries** | The document's issuing country (or the transaction's party country) is one of these |
| **Risk at least** | The item's risk is low, medium or high or more |
| **Workflows** | The item came through one of these workflows |
| **Amount from** / **Amount up to** | The transaction amount is in this range |
| **Warnings** | The item has one of these warnings or situations |
| **Face match below (%)** | The face match score is under this value |

Then choose a lower level, a higher threshold (80%, 85%, 90% or 95%), or both. The first segment that matches applies, and a segment can never raise a queue above its own level. An item that lacks a fact, for example a verification with no amount, never matches a condition on it. You can have up to 20 segments.

Press **Check impact** before you save to see how many waiting reports each segment and lock would apply to now. Backtests use your current segments and locks too.

## Locks

**Never decide alone if…** adds your own locks on top of the five fixed guardrails. Each lock has conditions like a segment's and one of two actions:

- **Never auto-approve**: a matching item can still be declined or sent for more information automatically, but never approved without a person.
- **Never decide alone**: a matching item always goes to a person.

A lock applies to the queues you pick, or to every queue. You can have up to 20 locks. When a segment or a lock changed what happened to an item, the report names it.

## Dashboard and Performance

**Dashboard** shows what the automation does for you. In shadow mode it shows what the Investigator would have done:

| Metric | What it means |
|---|---|
| **Would have decided** | How many reviews it would have decided on its own in shadow mode |
| **Agreement with your team** | How often it agreed with what your team decided, and how many disagreements there were |
| **Hours it would have saved** | Reviewer time, at your team's pace |
| **Moves to Auto** | The date it moves to **Auto** if agreement stays at 98% or more, or **When you decide** when automatic promotion is off or paused |

Once it decides its first item, the dashboard covers the last 30 days:

| Metric | What it means |
|---|---|
| **Automation rate** | How many of the reviews it decided on its own |
| **Reviewer hours saved** | At your team's median handling time, also counted in full-time reviewers |
| **Investigator spend** | What its reviews cost this month |
| **Cost per decision** | What its decisions cost, each |
| **Accuracy on the QA sample** | How many of the Full auto decisions a person checked agreed with it |
| **Overrides** | The share of its decisions a reviewer changed, with their reasons |

Below them, **By queue** splits these per queue, **Automation rate by week** shows each week (dashed weeks were in shadow mode), and **Overrides and their reasons** lists why reviewers changed its decisions. Overrides come back to your team as questions.

**Performance** compares the Investigator with your reviewers over the last 7, 30 or 90 days:

| Metric | What it means |
|---|---|
| **Agreement with reviewers** | How often a reviewer accepted the recommendation, as proposed or with edits |
| **Items investigated** | The sessions, transaction alerts and cases it reported on |
| **Time saved** | The median time to decide an item, before and now |
| **Overrides** | The share of decisions that overrode it, each with a reason |
| **Confidence you can trust** | How often reviewers agreed with it at each confidence it stated. High confidence items are eligible for batch accept, moderate ones are reviewed one by one, and low ones always go to a person |

It also charts **Agreement and volume by week**, splits the numbers **By queue** with the time to decide, lists **Overrides by reason**, and shows **Where reviewers disagree most**. Drift, consistency over re-runs and the QA of your reviewers, below, are on the same tab.

## Consistency over re-runs

A good reviewer gives the same answer to the same case twice. **Consistency over re-runs** in **Performance** measures that: it reviews a sample of recent items again, three times each by default, and shows how often the Investigator repeated its recommendation. It runs daily and whenever you press **Run check**, in shadow and at no charge. Below 90%, a consistency warning is written to **Activity** and, if you set one up, posted to your Slack or Teams channel.

## Drift protection

Drift protection compares, every day, how often the Investigator agreed with your team over the last 7 days against the 30 days before, and whether the kind of items it receives changed. When agreement drops by 5 points or more, or the mix of items shifts, the queue is marked **Drifting** and, with drift protection on:

- its level steps down one tier (**Full auto** to **Auto**, **Auto** to **Suggest**), at most once a week;
- your team is alerted, and the change is written to **Activity**;
- you raise the level again yourself in **Settings** once agreement recovers.

The quality sample is split between items where the Investigator and your team agreed and items where they didn't, so a drift shows up in the sample early.

## Service levels and priority

In **Service levels**, set how long an item may wait for a decision in each queue: 4 hours, 8 hours, 1 day, 2 days, 3 days or 1 week, or **No SLA**. Each waiting report then shows when it's due, and is marked **Due soon** in its last 4 hours and **SLA breached** after that. A breach can notify your team and send a webhook. Cases keep their own due date.

Every waiting item also gets a **priority** from its risk, the transaction amount, whether the person contested a decision or answered an information request, plus the time it has been waiting. Sort the queue by **Priority** to work the most urgent items first.

## QA of your reviewers

The Investigator can also check your team's decisions. Every day it takes a sample of the decisions people took in the last 7 days and scores each one for compliance with your SOP, the evidence used and the justification written. It also looks back over recent approvals for risks that were missed. A decision fails when its score is under 70 out of 100, when an approval missed a risk, or when it went against an SOP rule with no reason given.

A failed decision opens a reopen task in your [Inbox](https://help.didit.me/investigations/inbox) so someone can take another look, is written to **Activity**, and can send a webhook. QA reviews run in shadow: they never change a decision and are never billed.

**QA of reviewers** in **Settings** shows whether the daily QA is **On** for your application; if it's off, contact Didit to turn it on. **Run a QA pass now** scores a **Sample** of 10, 20, 50 or 100 decisions closed in the **Lookback window** you pick, the last 1, 7, 30 or 90 days, and looks back over their approvals. The results are in **Performance**: filter them by **Result** (**Failed** or **Passed**) and by **Kind** (**QA**, **Lookback** or **Both**). Each result opens a **Scorecard** with its **SOP compliance**, **Evidence** and **Justification** scores, and links to **Open the report** and **Open the reopen task in Inbox**.

## Validation report

Before a bank or an auditor lets an AI agent decide, they ask how it was tested. The **validation report** answers that for one policy version, as a PDF or as data:

- the policy version, and the Didit Investigator version and engine the reports were produced with;
- the Quick check and the AI backtest, with their agreement, disagreements and sample warnings;
- consistency over re-runs, and calibration: how often each stated confidence was right;
- the guardrails, your locks, segments, levels and amount cap;
- QA results, overrides and drift;
- what changed since the previous version, explained in plain language, with its measured impact.

## Activity and provenance

**Activity** keeps a permanent log of everything the Investigator and your team did: reports, decisions, overrides, questions, images revealed, notes, re-reviews, information requests, contests, consistency warnings, drift, SLA breaches, QA reviews and regulatory report drafts. Each entry made through Didit Copilot or MCP says so. Every report and entry carries its provenance: the Didit Investigator version and engine, and your policy version.

Filter the log by **Event type**: **All**, **Agent**, **Decisions**, **Overrides** or **Autonomy and guardrails**, each with how many entries it holds. **By** shows what **Anyone**, **The Investigator** or one person in the log did, and **Period** is **Today**, **Last 7 days**, **Last 30 days** or **All time**. **Show more** loads older entries, and the exports below use the same filters.

Deleting a case also erases its Investigator reports and their entries, because they quote the people involved. **Activity** keeps one entry in their place: which case was deleted, who deleted it and how many reports went with it, never what they said.

**Export for auditors** exports the log with your filters applied, as a CSV or as a PDF audit pack. **Reports of this view (ZIP)** downloads one review record PDF for each report in the view, plus an `index.csv` with each report's fingerprint. A ZIP holds up to 200 reports: if your view has more, narrow the filters above it and export again. The review record, and how to check a copy of one with its fingerprint, are explained in [the Investigator](https://help.didit.me/investigations/investigator).