File sizes
| fixture | index.html (bytes) | manifest.json (bytes) |
|---|---|---|
| forge-silent | 6592 | 2658 |
| forge-assisted | 6592 | 2658 |
| sumi-manual | 7573 | 3468 |
Differences are from (a) markup formatting (manual breaks attributes across multiple lines for readability), (b) label_i18n density on manifest non-action elements.
Body / root attributes
| fixture | data-nac-plugin |
|---|---|
| forge-silent | calc |
| forge-assisted | calc |
| sumi-manual | calc |
Button-by-button decoration
Joined on the invariant onclick handler from calc.js. Each row is one logical button. Cells show data-nac-id + role / verb.
| onclick | visible | silent id | silent r/v | assisted id | assisted r/v | manual id | manual r/v |
|---|---|---|---|---|---|---|---|
clearAll() | C | calc.action.clear | action / clear | calc.action.clear | action / clear | calc.action.clear | action / clear |
backspace() | ← | calc.action.backspace | action / backspace | calc.action.backspace | action / backspace | calc.action.backspace | action / backspace |
percent() | % | calc.action.percent | action / percent | calc.action.percent | action / percent | calc.action.percent | action / percent |
setOp('/') | / | calc.action.divide | action / divide | calc.action.divide | action / divide | calc.action.op_divide | action / divide |
digit('7') | 7 | calc.action.digit_7 | action / digit_7 | calc.action.digit_7 | action / digit_7 | calc.action.digit_7 | action / digit_7 |
digit('8') | 8 | calc.action.digit_8 | action / digit_8 | calc.action.digit_8 | action / digit_8 | calc.action.digit_8 | action / digit_8 |
digit('9') | 9 | calc.action.digit_9 | action / digit_9 | calc.action.digit_9 | action / digit_9 | calc.action.digit_9 | action / digit_9 |
setOp('*') | x | calc.action.multiply | action / multiply | calc.action.multiply | action / multiply | calc.action.op_multiply | action / multiply |
digit('4') | 4 | calc.action.digit_4 | action / digit_4 | calc.action.digit_4 | action / digit_4 | calc.action.digit_4 | action / digit_4 |
digit('5') | 5 | calc.action.digit_5 | action / digit_5 | calc.action.digit_5 | action / digit_5 | calc.action.digit_5 | action / digit_5 |
digit('6') | 6 | calc.action.digit_6 | action / digit_6 | calc.action.digit_6 | action / digit_6 | calc.action.digit_6 | action / digit_6 |
setOp('-') | - | calc.action.subtract | action / subtract | calc.action.subtract | action / subtract | calc.action.op_subtract | action / subtract |
digit('1') | 1 | calc.action.digit_1 | action / digit_1 | calc.action.digit_1 | action / digit_1 | calc.action.digit_1 | action / digit_1 |
digit('2') | 2 | calc.action.digit_2 | action / digit_2 | calc.action.digit_2 | action / digit_2 | calc.action.digit_2 | action / digit_2 |
digit('3') | 3 | calc.action.digit_3 | action / digit_3 | calc.action.digit_3 | action / digit_3 | calc.action.digit_3 | action / digit_3 |
setOp('+') | + | calc.action.add | action / add | calc.action.add | action / add | calc.action.op_add | action / add |
digit('0') | 0 | calc.action.digit_0 | action / digit_0 | calc.action.digit_0 | action / digit_0 | calc.action.digit_0 | action / digit_0 |
dot() | . | calc.action.dot | action / dot | calc.action.dot | action / dot | calc.action.dot | action / dot |
equals() | = | calc.action.equals | action / equals | calc.action.equals | action / equals | calc.action.equals | action / equals |
Reading the table. 15 of 19 buttons are byte-identical across all three decorations. The 4 differences are the operator buttons -- Forge uses
calc.action.add, Sumi uses calc.action.op_add. The verb name (add) is the same in both, so the dispatch path is functionally equivalent.
Manifest elements
Joined on the logical key (id minus the calc. plugin prefix). (missing) means that fixture's manifest omits the element entirely.
| logical key | forge-silent | forge-assisted | sumi-manual |
|---|---|---|---|
action.add | calc.action.add action verb=[add] | calc.action.add action verb=[add] | (absent: uses action.op_add) |
action.backspace | calc.action.backspace action verb=[backspace] | calc.action.backspace action verb=[backspace] | calc.action.backspace action verb=[backspace] |
action.clear | calc.action.clear action verb=[clear] | calc.action.clear action verb=[clear] | calc.action.clear action verb=[clear] |
action.digit_0 | ... verb=[digit_0] | ... verb=[digit_0] | ... verb=[digit_0] |
action.digit_1 | ... verb=[digit_1] | ... verb=[digit_1] | ... verb=[digit_1] |
action.digit_2 | ... verb=[digit_2] | ... verb=[digit_2] | ... verb=[digit_2] |
action.digit_3 | ... verb=[digit_3] | ... verb=[digit_3] | ... verb=[digit_3] |
action.digit_4 | ... verb=[digit_4] | ... verb=[digit_4] | ... verb=[digit_4] |
action.digit_5 | ... verb=[digit_5] | ... verb=[digit_5] | ... verb=[digit_5] |
action.digit_6 | ... verb=[digit_6] | ... verb=[digit_6] | ... verb=[digit_6] |
action.digit_7 | ... verb=[digit_7] | ... verb=[digit_7] | ... verb=[digit_7] |
action.digit_8 | ... verb=[digit_8] | ... verb=[digit_8] | ... verb=[digit_8] |
action.digit_9 | ... verb=[digit_9] | ... verb=[digit_9] | ... verb=[digit_9] |
action.divide | calc.action.divide verb=[divide] | calc.action.divide verb=[divide] | (absent: uses action.op_divide) |
action.multiply | calc.action.multiply verb=[multiply] | calc.action.multiply verb=[multiply] | (absent: uses action.op_multiply) |
action.subtract | calc.action.subtract verb=[subtract] | calc.action.subtract verb=[subtract] | (absent: uses action.op_subtract) |
action.op_add | (absent) | (absent) | calc.action.op_add verb=[add] |
action.op_divide | (absent) | (absent) | calc.action.op_divide verb=[divide] |
action.op_multiply | (absent) | (absent) | calc.action.op_multiply verb=[multiply] |
action.op_subtract | (absent) | (absent) | calc.action.op_subtract verb=[subtract] |
action.percent | ... verb=[percent] | ... verb=[percent] | ... verb=[percent] |
action.dot | ... verb=[dot] | ... verb=[dot] | ... verb=[dot] |
action.equals | ... verb=[equals] | ... verb=[equals] | ... verb=[equals] |
field.display | calc.field.display field | calc.field.display field | calc.field.display field (+ label_i18n) |
field.history | calc.field.history field | calc.field.history field | calc.field.history field (+ label_i18n) |
region.main | calc.region.main region | calc.region.main region | calc.region.main region (+ label_i18n) |
region.pad | calc.region.pad region | calc.region.pad region | (absent: uses region.keypad) |
region.keypad | (absent: uses region.pad) | (absent: uses region.pad) | calc.region.keypad region (+ label_i18n) |
Divergences
Every place where one fixture diverges from another (id structure, manifest role, or verb name).
| kind | element | forge-silent | forge-assisted | sumi-manual |
|---|---|---|---|---|
| id | setOp('/') | calc.action.divide | calc.action.divide | calc.action.op_divide |
| id | setOp('*') | calc.action.multiply | calc.action.multiply | calc.action.op_multiply |
| id | setOp('-') | calc.action.subtract | calc.action.subtract | calc.action.op_subtract |
| id | setOp('+') | calc.action.add | calc.action.add | calc.action.op_add |
| shape | region.pad vs region.keypad | uses pad | uses pad | uses keypad |
| i18n | label_i18n on field+region | none | none | full es/en on every element |
Reading the divergences. Zero verb-name divergences. The 4 operator id divergences are stylistic (
action.add vs action.op_add) and don't change dispatch behaviour because both refer to the same verb. The only structural difference is the keypad region name (pad vs keypad) and Sumi's i18n labels on non-action elements. The verb-set the LLM has to choose from is identical in size for all three.
E2E driver cost (latest run)
Source: results/calc_e2e_2026-05-20T19-55-11-899Z.jsonl, 1 iteration per (fixture, task), driver Claude Sonnet 4.6.
| fixture | mean tokens in | mean tokens out | mean LLM latency (ms) | 5-task cost (Sonnet 4.6) |
|---|---|---|---|---|
| forge-silent | 1307 | 108 | 3288 | $0.02774 |
| forge-assisted | 1307 | 108 | 2651 | $0.02774 |
| sumi-manual | 1755 | 107 | 2772 | $0.03435 |
E2E success rate (latest run)
| fixture | tasks passed |
|---|---|
| forge-silent | 5 / 5 |
| forge-assisted | 5 / 5 |
| sumi-manual | 5 / 5 |
The headline. 19 of 19 button decorations agree on verb. Three of 19 disagree on id naming (operator buttons). One region label disagrees (
pad vs keypad). Sumi-manual adds label_i18n across the board which costs +17% input tokens per dispatch with no functional benefit for the LLM driver. All three pass every task in the suite.