LanguageEnglish

Tests

Open Workspace tools → Tests on desktop or Tools → Tests on mobile, then choose your project. Tests reads your project’s test files as text and shows every test it finds: folders, files, test names and the code around each test. Reading never runs, imports or installs anything; your tests only run when you choose Run.

The top of the page has a one-line verdict and two cards. Unit tests and Regression tests each show how many tests passed in their latest run, what failed or is unstable, the recent runs and a Run button. Below them, one tab bar lists Unit tests, Regression tests, Integration & E2E, Checks (lint, types, build and verification scripts) and History.

Read the verdict

The line above the cards answers first, from the latest results of your tests:

  • Not checked yet until a test has run. Run your tests once to see where your project stands.
  • When tests fail, it counts them and offers Show failures: the failing tests open with the Failed filter, starting with a regression test when a protected bug is back.
  • When tests are unstable (they failed, then passed on a retry), it counts them and offers Show unstable tests.
  • Otherwise it counts the tests that pass, how many have not run yet and when the last run finished.

Counts add unit tests, integration and end-to-end tests, and checks. Regression tests reuse those same tests, so they are not counted twice.

Next to the Automatic runs button, the line also shows Next automatic run with its time, or Automatic runs off. When the last automatic run could not start, it says why.

See your tests

Each tab groups test files by folder. Open a folder to see its files, then select a file: its tests appear on the right (on a phone, on their own screen), failing tests first. A failing test shows the expected and received values reported by its test runner and the current code around the failing line. Filter by Failed, Unstable or Not run yet, or search by file or folder name.

A test that has never run reads Not run yet. The pass ring never shows 100 % while a test failed. A state always has an icon and a word, not only a colour.

Tests sorts files automatically: unit tests, integration and end-to-end tests (folders such as e2e, integration, routes or api, Playwright files and web scenarios), and checks. Checks cover lint, types, build and your project’s verification scripts, including scripts named with a suffix such as contrast:ui-check or tabs:guard. If a folder is in the wrong family, open its ⋯ menu and choose Move to Unit tests or Move to Integration & E2E.

Run tests

Choose Run unit tests, Run regression tests, the run button of a folder or a file, or Run file in its details. You do not configure anything first: Tests builds the command from the script that already runs those files in your project (keeping its options) or from the test runner’s default, saves it, and starts it. The run report opens with its progress and Stop. Closing the app does not stop a run; the instance keeps the result.

While a set of tests runs, the verdict at the top of Tests shows how far it went, with Show progress, which reopens the run in progress, and Stop, which stops the whole set: runs still waiting never start. You can leave Tests and come back; the progress is still there. Run unit tests reads Running… until the set ends, so a second click never starts the same tests twice. When a test tool is missing on your instance, for example pytest, only its tests read not started; the others still run.

A saved command stays editable: ⋯ → Edit command changes its executable, arguments, level, prerequisites, services, test data and deadline.

Choose how many test files run at once

Test files at once, next to the verdict, sets how many test files of a project run at the same time. Automatic, the default, keeps each test tool’s own setting. Choose a number, up to your instance’s processor count, and every run of the project, manual or automatic, runs at most that many: Node, Vitest, Jest and Playwright follow it, other test tools keep their own setting, and the project’s runs then go one after another. A run already started keeps its number.

Tests always run at a lower processor priority than the instance, so your conversations and apps stay responsive while they run. Only you change this setting: the in-app assistant can read or change it at your request, and a project conversation never changes it.

Run tests on a schedule

Choose Automatic runs next to the verdict to run a project’s tests on its own schedule. Each project has one:

  • Run tests automatically turns it on or off.
  • Recurrence: hourly, daily, weekdays, weekly, monthly or a custom cron expression. Runs must be at least 15 minutes apart.
  • Time zone: the zone the schedule follows. Your display time zone is offered first; changing it later does not move a saved schedule.
  • What to run: at least one of Unit tests, Regression tests, Integration & E2E and Checks.
  • Notify me: Only when something fails (the default) keeps passing runs quiet, while a run that fails, is interrupted or needs review still notifies you; After every run notifies you each time.
  • Missed runs: when the instance was off at a scheduled time, Run once when the instance returns (the default) makes up for it with a single run, however many times were missed; Skip missed runs waits for the next scheduled time.

An automatic run is an ordinary run: it updates the cards and the verdict, and appears in History with its report. It never overlaps the previous automatic run. Under memory pressure or during an update, it waits and tries again for up to two hours; it is never counted as a failure. When an automatic run cannot start, for example because the previous one was still running, the project needs review or there was nothing to run, the verdict says why and the next scheduled time follows as usual. A kind of test with nothing to run does not stop the others. After an update or a restore, automatic runs resume on their own; a time missed meanwhile follows your Missed runs choice.

Only you set a schedule: the in-app assistant can read or change it at your request, and a project conversation can read it but never creates or changes one.

Create a test

Choose New test at the top right:

  • Write a test with AI: describe a behavior; AI proposes the test and you review it before it is saved. Nothing runs until you run it.
  • Protect a fixed bug: create a regression test from the tests that guard it.
  • Web scenario: steps clicked and checked in the integrated browser.
  • Add a test command: an existing script of your project, such as npm test.

Edit a test

Select a file, then Open in editor to change it yourself, or Edit with AI to open a new conversation about this file and test (nothing is sent before you send it). When a test failed, Fix with AI reproduces the failure, corrects the code and replays the same test and your regression tests. A regression test’s Edit tests adds or removes the tests that protect it.

Run Regression tests

Name a collection, describe its protected behavior or bug, and select exact saved suite, case or scenario revisions. Unit, integration, API and E2E tests can share a collection without duplicated sources. Lint/types/build remain separate checks.

Save, then explicitly run that revision. Full and partial selections are labelled separately. Overlaps execute once; conflicting revisions and empty selections are refused. Later collection edits do not change a Run’s captured membership.

Collections remain private instance state, without a Git export. Existing project test files and explicitly exported scenario sources can be versioned normally.

Fixed tests become regression tests

In every project, a bug is protected as soon as its fix is proven, without anyone keeping a list. When a test that failed passes after the code changed, Tests turns it into a regression test:

  • A test counts as fixed when it passes on every attempt of a run and its most recent result in the project’s five previous runs was a failure on different code. A pass after a retry, or on unchanged code, is not a fix.
  • Each fixed file becomes one regression test, named after its first fixed test, that runs that file alone with the options of your project’s script. Its protected behavior lists the fixed tests.
  • A run that fixes more than five files protects nothing: that many fixes at once usually mean the environment was repaired, not that bugs were fixed.
  • A file that a regression test already protects is left as it is.

Nothing runs when it is created: the next regression run, yours or an automatic one, includes it. Edit or remove it like any other regression test.

Choose a reference and compare

Choose a retained Run as the collection’s reference, then compare another Run. A reference is a comparison baseline, not a third execution-source mode. Success never replaces it automatically.

A reference needs nonempty, fully observed assertions, known expectations, isolated data and complete provenance. A failing baseline can identify existing failures; partial, imported, skipped, unstable or incomplete evidence is ineligible.

Comparison distinguishes new regressions, existing failures, fixes, additions, removals, skips, unexecuted cases and instability. It reports changed expectations and configuration/data/runtime/target compatibility. Partial absence is not removal. Complete comparison evidence can include failures; it does not mean passed.

Add a web scenario

Edit a scenario’s behavior, target, data, prerequisites, steps, assertions and cleanup. Use stable role/label/test ID/text locators. This is not a click recorder.

For example, protect a task application’s persistence with these steps:

  1. Open the application using dedicated test data.
  2. Fill the new-task field with a known title and select its add action.
  3. Assert that the task is visible, then mark it complete.
  4. Reload the page.
  5. Assert that the same task is still visible and checked.

Save the scenario before running it. Export source is another explicit action: choose a project-relative file path to retain the generated test source. Export refuses existing different bytes. Editing an exported file invalidates its mapping and blocks replay. Review the scenario revision and its source mapping; do not overwrite concurrent work. Earlier proof remains historical evidence. Test preparation does not change ordinary application Start, Stop or activation.

The integrated browser uses ephemeral Chromium contexts and blocks service workers. It does not qualify offline/PWA behavior, Safari, Firefox, native Android/iOS behavior or real-device performance. Current target-to-source correspondence can remain uncertain: successful browser assertions then read Passed with that fact in Details, but the Run is not a complete validation or an eligible full reference. A screenshot is evidence; visual regression requires an executed assertion and an explicitly managed visual baseline.

Inspect failures and keep the evidence

A Run’s report opens on its answer: Passed, Failed, Unstable, or one sentence saying why it gave none, with the name of what ran, its number of tests, its duration and its date. Failing tests come first, with the expected and received values when the engine supplied them; Show log opens the log, and Details holds notes (files that changed while it ran, tests that used the project’s own data), saved files, how it ran and the comparison with another run. Logs and lists are bounded and paginated; truncation stays visible.

Command success, verdict, report completeness and provenance are separate. Exit zero without a usable report proves command success only. Zero cases, missing reports/authentication/dependencies, skips and interruptions cannot validate. A failed attempt followed by a pass remains flaky, with both retained.

Captures, traces and reports retained by Tests survive their originating browser session. Downloads require instance authentication; active HTML/SVG is downloaded as an attachment, not executed as a report. Missing or damaged evidence is named. Opening source resolves the current project file, not a promise that its bytes still match the historical test. Later edits do not rewrite earlier results.

Stop waits for owned children and preserves unrelated applications and conversations. A refused cleanup remains visible. Closing the screen, losing the network or backgrounding the client does not Stop the Run. Reopen its exact report after reconnecting. Instance restart interrupts unproven continuity and does not automatically replay a command. Runs and notifications link back to the retained Tests result.

Turn a bug into a protected behavior

Keep the chain: reproduce → retain a red test → correct the code → same expectation green → replay the regression collection. Replays create linked records. Changing, deleting or weakening assertions does not automatically resolve a defect.

You can write tests yourself or choose Describe a behavior for an editable unit-test or scenario proposal. Select the existing Harness, Model, Effort and account controls. Review concrete assertions, files, data and prerequisites; complete unanswered questions before explicitly marking a review ready and saving. Generation, saving and running remain separate actions. Saving a reviewed unit proposal writes its project test file; saving a scenario stores its definition, with optional source export as another action. A proposal has no executed verdict. Source changes can require a new review before saving.

Fix with AI links the exact failure to reproduction, a visible correction conversation and new verification. Missing reproduction, changed expectations and incomplete regression evidence stay explicit. A model’s explanation never replaces assertions. Local tests need no model; optional AI uses existing provider usage, pricing and quota controls. Missing cost is unknown, not zero. See Usage accounting. Nothing invokes Ship or publishes.

Qualified engines and prerequisites

These exact versions have Linux source qualification with Node 22.22.2/22.23.2. Dependencies must exist; Tests does not install or upgrade them. No version range is implied.

Engine Cases and targeted replay Coverage and limits
Node test runner 22.22.2 / 22.23.2 Structured cases, hooks, skipped/todo; direct current Node executable. No exact case rerun or retry-history capability. Explicit native line/branch/function coverage with its measured perimeter; bounded to 128 files/64 KiB of coverage metadata.
Vitest 4.1.10 Direct entrypoint; exact retained-case selection after supervised collection; attempts preserve retries. No qualified coverage or expected-failure inversion. Lifecycle wrappers and configurations that cannot retain the evidence plugin are refused.
Jest 30.2.0 Unique full name in a pinned file; one selected case per command, with bounded commands for several cases. No coverage import. Ambiguous names, presets, multi-project/TypeScript configuration and custom runners are refused.
pytest 8.4.2 / CPython 3.10.12 Direct virtual-environment Python -m pytest; exact node identities and setup/call/teardown. No coverage, wrapper, distributed or retry-plugin qualification. Missing pytest is not silently repaired.
Playwright Test 1.63.0 Installed project runner; project/repeat identities, retry attempts and attachments. No exact case rerun. No coverage or expected-failure inversion. Its project browser policy is distinct from the integrated browser.
Custom command / JUnit Opaque command outcome or cases actually supplied by a bounded report. No exact case rerun. No guessed coverage, counts or retry interpretation; unsupported XML dialects remain invalid/incomplete.

A custom JUnit command must write its current report into the private Run report directory using TEST_REPORT_PATH / TEST_REPORT_DIR or their explicit argument tokens {{TEST_REPORT_PATH}} / {{TEST_REPORT_DIR}}. An old workspace report cannot validate a new execution. External XML import is available through the API/action tests.importJUnit, not an import button in the current interface. Imported counts do not establish local execution, current provenance or a reference.

Linux execution is qualified. A Windows ownership path exists, but native Windows execution remains unqualified. Execution on a macOS instance is currently unsupported; a macOS client can control a supported remote instance. Android/iOS interface availability does not certify native application testing. Source, exact payload, private installed candidate, served client, public release and physical-device evidence are separate; one does not establish the others.

Conversation tools

Ordinary authorized project conversations expose five MCP tools and matching CLI commands: aircode_tests_catalog / catalog, aircode_tests_describe / describe, aircode_tests_run / run, aircode_tests_status / status, and aircode_tests_open / open. Read a schema before a mutation:

aircode-tests catalog
aircode-tests describe --input '{"action":"tests.launchUnits"}'
aircode-tests run --input '{"action":"tests.units","input":{}}'

Use tests.launchUnits for unit suites or tests.launchRegression for an exact collection, with the returned scope/revision pins and the schema’s explicit key. The session capability supplies the project; do not add a project field. --input - accepts bounded JSON on standard input. status takes {scopeId,ownerKind,ownerId}; open takes the same identity or {} for Tests. Keep its returned openPath unchanged. For MCP, use the same arguments with the matching aircode_tests_* tool. See Documentation for AI.

Conversation launches accept only recipes and services recognized by discovery. Use a recognized project script, or launch a custom command from the owner interface.

After a lost response, inspect status and reuse the same key and input for an explicit identical retry. Reading status never relaunches work. Planning and read-only sessions cannot write; expired or revoked tools require normal conversation resumption. These tools grant no general shell or public evidence transfer. Do not edit a store or manufacture credentials to bypass a refusal.

Storage and actionable refusals

Definitions, references and immutable execution facts live in the instance’s private Tests SQLite domain. Large logs, captures and traces are private companion files. Database backups alone do not contain those bytes: preserve companions with a consistent instance backup. Never copy a live WAL database as a backup. Unique/core evidence is retained by default; only a verified redundant terminal attachment may be explicitly removed. There is no general purge of unique proof. Recognized secrets are reduced or refused; filters cannot identify every secret inside arbitrary logs, images, archives or project data.

Refusal or attention state Next action
No unit suites / empty collection Classify and save suites, or add exact collection members.
Missing or unsupported engine Review the executable, installed dependency and qualified version; use an honest custom command if appropriate.
Data not isolated / uncertain source The report reads Passed with the fact in Details; establish dedicated fixtures before such a Run can become a reference.
Targeted replay unavailable Rerun the full selection or inspect its retained report.
Source changed / incomplete reference Review current sources, expectations and compatibility, then explicitly choose the appropriate new selection or baseline.
Queue, memory, disk or quota deferral Resolve the named resource constraint and request another launch or resume explicitly. No reset time or retry is invented.
Cleanup refused / evidence missing Inspect the retained reason and evidence; use available evidence recovery without replaying tests or manually deleting private state.