574 / 600 at N=10.
Four models × five languages.

Conduit is what the RealWorld project ships as the canonical app for comparing frameworks: a Medium-style blogging site with user auth, articles, comments, follow/favorite, tags, profile + settings pages. Real surface, real complexity, used to evaluate ~24 frontend frameworks in head-to-head form. We picked the community fork that comes with both a React/Vite frontend AND a hand-written Cypress test suite, so the comparison Forge-vs-manual was apples-to-apples in the same codebase.

--ai-apply is the JSX counterpart to --ai-silent. Walks every .jsx / .tsx under <repo>/src, sends each to Claude Sonnet 4.6 in parallel (concurrency 4 by default, max 8), receives back the decorated JSX plus a manifest fragment listing what was decorated. Forge aggregates fragments into a single manifest.json at the project root.

# From a fresh clone of the Conduit fork
yf migrate ./conduit-forge-ai \
  --ai-apply \
  --ai-plugin-slug conduit \
  --subdir src \
  --ai-out ./conduit-forge-ai \
  --ai-concurrency 4

# Output:
AI apply complete.
  Files scanned:   47
  Files decorated: 34
  Manifest elements: 59
  Plugin slug:     conduit
  Tokens in/out:   43428 / 22192
  Total cost:      $0.4632
  Total LLM time:  262s wall (across 47 files, divided by concurrency = ~70s real)

Per-role breakdown of the 59-element manifest:

Quality assessment: IDs are semantic (not ordinal like btn_3), verbs are imperative (publish, follow, favorite, delete), roles match content (display/history-style fields as field, navigation links as navigation, feed switchers as tab).

yf gen-tests takes the Forge-aggregated manifest and emits an e2e suite in whichever test framework the user's project carries. Multi-framework on purpose: most adoptions already have Playwright or Cypress or Vitest installed -- Forge does not mandate a choice.

yf gen-tests ./conduit-forge-ai --framework all --base-url "http://localhost:3000"

# Output:
Manifest elements:  59
Test cases:         66                          # some elements have multiple verbs
Frameworks:         playwright, cypress, vitest, wdio

playwright: + tests/forge-e2e/conduit.spec.ts
cypress:    + cypress/e2e/forge/conduit.cy.js
vitest:     + tests/forge-e2e/conduit.test.ts
wdio:       + test/specs/forge-e2e/conduit.test.ts

Coverage: 59 / 59 manifest elements exercised.

The coverage metric is "agent-relevant surface coverage" -- one test case per (nac_id, role, verb) tuple in the manifest. This is the honest claim; we do not report this as line-coverage (which the manifest alone cannot guarantee).

The fork's testAutomation/ directory carries 14 hand-written Cypress cases across 3 .cy.js files. Forge's emitter produces 66 cases in one .cy.js. The comparison is structural here -- counting cases + verifying surface coverage -- because operational pass-rate validation requires running both suites against a live backend.

We stood up the Conduit backend (SQLite via Sequelize dialect swap), brought up the migrated Vite frontend, and ran both Cypress suites end-to-end. The structural delta (66 vs 14) holds; the operational delta tells a different story we publish unsoftened.

Two structural conclusions hold despite the operational gap:

1. Forge's 59-element manifest is correct: NAC3 attached, plugin registered, every decorated element discoverable via NAC.click_by_verb when its containing route is mounted.
2. The SPA-aware nac-bridge (MutationObserver) works: late-mounted action/navigation/tab elements get bound automatically as routes change.

Ran a Pilot-style operational demo: tell Sonnet 4.6 "log in as sumi@yujin.app", give it the live manifest, let it pick verbs turn by turn, dispatch via NAC.click_by_verb. Every dispatch returned OK (the bridge fires the contract event), but the URL never changed. The model picked conduit.navigation.nav_item verb navigate six turns in a row -- each click hit the same DOM element (the first NavItem) which happened to be the Home link, not the SignIn link.

Diagnosis: Conduit's Navbar uses a <NavItem> component reused for Home, SignIn, SignUp, Settings, Profile. --ai-apply sees the component once, decorates the JSX with data-nac-id="conduit.navigation.nav_item", and that single ID propagates to every rendered instance. The manifest deduplicates by ID, so the 5+ DOM instances collapse into 1 manifest entry. NAC.click_by_verb then matches the first DOM occurrence -- always the same one, regardless of which the user actually wanted.

Closed finding 04c at the architectural level. --ai-apply now infers per-instance qualifiers from JSX props (label, name, to, slug, etc), bakes them into data-nac-id as a __qualifier suffix, and emits N manifest entries per known instance (or instance_pattern for runtime-dynamic qualifiers). The NAC3 runtime's click_by_verb now accepts { qualifier } and { qualifier_ordinal } in opts for explicit instance targeting.

Verification on Conduit v2 (re-decorated post-H2):

Closed Hallazgo 1 at the architectural level. --ai-apply now extracts a route_map from any file that is a React Router root, and each manifest element carries mounted_at: string[] + requires_auth: boolean. The Cypress emitter consumes both: every test does cy.visit(mounted_at[0]) before dispatch, and auth-gated tests insert a loginViaApi() pre-step (drops a JWT in localStorage, no UI navigation needed).

Per the brief's communication discipline: gen-tests is not communicated publicly as a finished product until pass-rate is ≥70% sustained. This case study documents the H1 architectural unlock + the honest 22× lift, NOT a sales claim. The "agent-relevant surface coverage" metric (every (nac_id, role, verb) tuple) remains the only quantitative claim safe to publish; the test-pass-rate is documented as an honest WIP signal here.

Two fixes that together destrabaron el demo Pilot end-to-end:

1. Pilot per-turn state probe now includes the current value of every visible input/textarea (keyed by data-nac-id + fallbacks), plus the navbar text snapshot, plus the last 5 action_history entries. System prompt got a STATE AWARENESS section: "before you emit a fill, check if the field already has the target value -- if it does, ADVANCE."
2. NAC.fill is now React-safe. The previous implementation set el.value = newVal directly. React silently reverts that on the next render because its synthetic event system tracks the original property descriptor on HTMLInputElement.prototype. The fix: use Object.getOwnPropertyDescriptor(HTMLInputElement.prototype, 'value').set.call(el, newVal). Same pattern for textarea / select / checkbox. Plain DOM behavior unchanged. (Committed in rpaforce-crm@c1bd8dc5.)

Conduit gen-tests (Cypress) and the Pilot agent (Sonnet / Haiku / Flash-Lite / gpt-5-mini) read the same manifest but ask different questions of it. gen-tests asks "is every element reachable?". Pilot asks "can a model drive a multi-step flow?". The numbers split. The closure-to-90 work (§04h below) is what brings gen-tests from the 33% checkpoint to a final 91.1% peak / 89.3% median.

Re-corri la suite Cypress generada por Forge contra la Conduit live despues de H4.pre + el fix de React-safe NAC.fill, esperando un lift sobre el 33% pre-H4. El numero quedo en 23/70 = 33%. Dos fixes que parecian deberian moverlo no lo movieron, y la razon es clara + honesta:

The 33% checkpoint at §04e was the H1 route-aware unlock. Closure-to-90 bundled a series of layered fixes that lifted the structural pass-rate from there to median 50 / 56 = 89.3%, with peak 51 / 56 = 91.1% and floor 48 / 56 = 85.7% across 3 consecutive runs. Net lift on the structural metric: 30.7% → 91.1% peak, +60.4 points. Trajectory:

Determinism across 3 consecutive runs of the full close-to-90 lot: 51/56 (91.07%) / 50/56 (89.28%) / 48/56 (85.71%). Median 50/56 = 89.3%. The 90% threshold is touched but not stably cleared. Variance source: the multi-user fixture's USER1 / USER2 login race in cy.request resolution timing -- 1-3 secondary-author follow / favorite tests flake when Cypress.env hasn't yet swapped to USER1 when the it() body runs.

# 1. Clone Conduit + install
git clone https://github.com/digitalinnovationone/conduit-realworld-example-app-with-cypress-automation conduit
cd conduit
npm install

# 2. Decorate with Forge (you need a paid seat + ANTHROPIC_API_KEY)
export ANTHROPIC_API_KEY=sk-ant-...
cp -r ./frontend ./frontend-forge-ai
yf migrate ./frontend-forge-ai \
  --ai-apply --ai-plugin-slug conduit --subdir src --ai-out ./frontend-forge-ai

# 3. Generate the test corpus in whatever framework you want
yf gen-tests ./frontend-forge-ai --framework cypress
# or --framework playwright,vitest,wdio,all

Total cost to reproduce: ~$0.50 in Anthropic tokens. Total wall time: ~3 minutes including npm install.

Everything in this case study runs on three pieces of open infrastructure: NAC3 v2.3.1 (the protocol, Apache-2.0), Yujin Forge (the build tool, paid seat), and Yujin Pilot (the embedded driver, open core). All three ship today.