tranquil
Docs

Smoke Tests

The tranquil-test-suite repo holds deterministic GUI smoke tests that run without an LLM in the loop. They launch a fully isolated Tranquil instance, drive it over the Chromium DevTools Protocol (CDP), assert real UI behaviour, and tear the app down — so regressions in shipped features are caught mechanically.

They are the automated counterpart to Agent Debugging (interactive/LLM-driven CDP inspection) and to tranquil-client’s yarn verify (interactive isolated launch). Same isolation contract, run as pass/fail assertions.

What they cover

Ports of the checks first verified by hand against a live build:

  • ADR-0012 — the vertical tab panel: lives in the right dock, mirrors the center pane’s tabs, file octicons (icon-markdown / icon-code), click-to-activate, live update as tabs open, toggle removes/restores + reveals the dock, close buttons.
  • ADR-0013 — business-theme ui-variables resolve to dark values (dark background, light text — not the light fallback), and no button{…!important} rule is loaded.
  • Settings default tab — Settings opens on the Tranquil tab, pinned first in the menu.

Running

cd tranquil-test-suite
deno task test

Requirements:

  • Deno (installed at ~/.deno/bin/deno).
  • A sibling ../tranquil-client checkout with node_modules installed. Electron is spawned directly — no Node is invoked.
  • A display: the suite opens a real window (fine locally on macOS). Headless CI would need a virtual framebuffer (e.g. xvfb) — not set up yet; the run model today is on-demand, whenever you touch UI or owned-package code.

A green run reports three ok steps and exits 0; any failed assertion fails its step and exits non-zero. Set SMOKE_DEBUG=1 to see the launched app’s stdout/stderr.

How it’s built

tranquil-test-suite/
  deno.json                       # tasks.test, @std/assert + @std/path
  smoke/
    main_test.ts                  # Deno.test: launch → run suites as steps → teardown
    lib/launch.ts                 # isolated Electron launch (Deno.Command) + teardown
    lib/cdp.ts                    # CDP over Deno's native WebSocket
    atom-home/config.cson         # FIXED config (theme pinned) for determinism
    suites/adr-0012-vertical-tabs.ts
    suites/adr-0013-ui-variables.ts
    suites/settings-default-tab.ts
  fixtures/                       # project content the suites open
  • lib/launch.ts mirrors tranquil-client/scripts/verify.js. It finds the Electron binary via node_modules/electron/path.txt (no Node), seeds a throwaway ATOM_HOME from smoke/atom-home/config.cson, and spawns Electron with a throwaway --user-data-dir, ATOM_HOME/ATOM_RESOURCE_PATH/NODE_PATH set, ELECTRON_RUN_AS_NODE deleted, and TRANQUIL_VERIFY=1. That env var activates the gated filter in src/main-process/atom-application.js (openPaths) which drops the tranquil-client resource-path root, so only tranquil-test-suite opens as the project — no flash of the app’s own source. teardown() sends SIGTERM to the Electron main (SIGKILL fallback) and removes the temp dirs.
  • lib/cdp.ts — a tiny CDP client: connectAtom() finds the editor window (the page whose atom global has a loaded project), evalInAtom<T>(expr) runs JS and returns its value (awaiting promises), waitFor(expr) polls until truthy (used instead of fixed sleeps).
  • atom-home/config.cson pins core.themes to tranquil-business-dark so theme/colour assertions are stable across machines (the real ~/.tranquil doesn’t set it).

Determinism

  • Fixed config for the theme; on a dev machine the harness also symlinks ~/.tranquil/dev so owned packages load exactly as in interactive use (bundled fallback otherwise).
  • Poll-based waits, never fixed sleeps.
  • Semantic colour assertions (dark bg / light text) rather than exact RGB — robust to minor theme tweaks, still catches the light-fallback regression.
  • Subset assertions on rows/tabs — the app auto-opens some panes, so suites assert their fixtures are present rather than an exact list.

Adding a smoke test

  1. Add smoke/suites/<name>.ts exporting export async function <name>(cdp: Cdp): Promise<void>; use cdp.evalInAtom / cdp.waitFor and @std/assert. Drive the app into a known state first.
  2. Register a step in smoke/main_test.ts: 
    await t.step("<name>", () => yourSuite(cdp));
  3. If it needs new content, add tracked files under fixtures/.
  4. deno task test (and deno fmt / deno lint / deno check) should stay green.