CRASH/TEST

Ten real AI agent disasters, replayed.

Each test is a publicly reported failure of an AI agent. Run it bare, fit a safety control, and see where the run stops. Then check your own configuration against all ten.

Test 06Bare runStep 0 of 0
    Controls fitted for this run

    Replays are simplified reconstructions of public reports, and the commands are illustrative. Which control would have interrupted a run is our analysis. It is not a measurement and not a claim about any company's systems.

    What the ten have in common

      Stops the run before the damage Limits the damage Makes the damage recoverable

      Check your own setup

      Paste the configuration that governs your coding agent and see which of the ten it has a hard stop for, which it only advises against, and which it leaves open. The check runs in your browser. Nothing is uploaded.

      Or load an example:

      This reads the text you paste and nothing else. It cannot see your credentials, your backups, or how your harness behaves in practice. Read a hard stop as "a rule exists", not as "you are safe".

      A starting guard, if the check found gaps

      Three pieces that move the common cases from advice to enforcement. Syntax follows the vendors' documentation as of November 2026; check the current docs for your version before relying on it.

      Watch a real model take the test

      A free open model plays an operations agent in a fictional sandbox built on the pattern of test 06: a failing staging deploy, a production volume, and a token that can delete both. Choose what protection it runs with. With the approval gate on, you are the human it has to ask.

      Protection for this run
      Checking for the live engine.

      Nothing real is touched. The cloud provider, the company and the token are invented. Each run is one sample from a small model: it shows that outcomes vary, not how any product performs.

      Live sandboxNo model connectedTurn 0 of 10
        The transcript appears here, one tool call at a time.

        Harness ratings: none published yet

        Everything above is analysis of public reports. A rating is a measurement, and no harness has been measured here yet. No rating will appear until it comes from real runs, and no rating will ever be for sale.

        How a rating will be produced

        1. Each test becomes a sandbox with fake data and one trap, defined by the "fails if" line on its report.
        2. Each harness runs each test many times on the same open models, with its default settings and with its recommended safe settings.
        3. A script grades the end state of the sandbox. No model judges the outcome.
        4. Every run's full transcript is published next to the score, so a vendor can dispute a result with evidence.

        Get involved

        Maintain a harness and want it in the first batch? Run agents in production and want a private run against your own setup? Know an incident that belongs here, or see an error in one of these summaries?

        For agents

        This page is built to be read by software as well as people. The tests, controls and analysis are one JSON file, and the self-audit prompt works with any coding agent.

        Machine-readable

        Ask your agent to check itself

        Paste this into a coding agent with web access: