← Field Notes
Field Notes · · Frames

How often do Frames actually render?

200 build prompts, one metric that matters: does the thing show up on screen.

Claim

Across 200 representative build prompts, Frames produced valid, parseable HTML 96.0% of the time, and rendered something usable on the first try 93.5% of the time. The gap between those two numbers is the interesting part.

Frames is the panel where Wren's output stops being text and starts being a thing you can click. So the honest question isn't "is the code good" — that's subjective and slow to score. The honest question is narrower and testable: when a user asks Wren to build something, does it render? A page that throws a parse error before the first paint has failed the user regardless of how elegant the intent was.

Method

We assembled 200 build prompts meant to mirror what people actually type into Frames, sampled from three buckets in rough proportion to real usage:

  • Single-file UI (120 prompts). "A pricing table," "a countdown timer," "a resume in one page" — self-contained HTML/CSS with a little JavaScript.
  • Interactive / stateful (55 prompts). "A to-do list that saves," "a tip calculator," "a quiz with a score" — anything with event handlers and mutable state.
  • Data & SVG (25 prompts). "A bar chart of these numbers," "an SVG badge," "a sortable table" — output where a single malformed tag breaks the whole view.

Each prompt ran once, on Wren Field (the default tier), with no retries and no human touch-ups — because that's the experience a first-time user gets. We then scored two things automatically:

  1. Valid HTML. The output was run through a strict parser (the same DOM parser the Frames preview uses) and flagged if it raised a fatal error or produced a document with unclosed structural tags.
  2. First-render success. The Frames iframe was loaded headless and screenshotted after 1.5s; a note passed only if it painted non-empty, non-error content without an uncaught console exception.

"Valid" is a lower bar than "good." A page can parse cleanly and still be ugly or subtly wrong. We are deliberately measuring the floor here, because the floor is what determines whether the product feels reliable.

Data

The headline numbers, by bucket:

BucketPromptsValid HTMLFirst render
Single-file UI12098.3%96.7%
Interactive / stateful5594.5%89.1%
Data & SVG2588.0%88.0%
All prompts20096.0%93.5%
Single run per prompt on Wren Field, no retries. Valid HTML = parses without fatal error; First render = paints usable content within 1.5s headless.

So 8 of 200 outputs (4.0%) failed to parse, and 13 of 200 (6.5%) failed to render usefully on the first try. Every render failure that was not a parse failure — 5 of them — came from JavaScript that threw at load, not from broken markup. That's the shape worth naming: markup is mostly solid; the fragile part is behavior.

The failures cluster into three modes:

  • Unclosed tags on long outputs (5 of 8 parse failures). The single most common defect. On longer generations — past roughly 220 lines — a closing </div> or </table> occasionally goes missing near the end, and the browser's error recovery reflows the layout into something wrong. Length is the strongest predictor of failure we found.
  • Runtime JS errors (5 of 13 render failures). A reference to an element that isn't in the DOM yet, or a typo'd variable in an event handler. The page parses fine and then goes blank on interaction.
  • Empty or off-viewport paint (3 of 13). Content that renders but with absolute positioning or a zero-height container that leaves the visible frame blank — technically "there," practically not.

Limitations

This is a small, one-shot, self-run study, and the number is softer than one decimal place implies. Specifically:

  • n = 200, single run. With no retries, the 88% on the 25-prompt Data & SVG bucket rests on just three failures — swing one prompt and it moves four points. Treat sub-bucket numbers as directional, not precise.
  • Our prompts, our scoring. We wrote the prompt set and the pass/fail rules. A different distribution of asks — heavier on charts, or on multi-file apps we don't yet support — would score lower.
  • "Valid" ≠ "correct." A page that renders can still misread the request. This note says nothing about whether the output did what was asked; it only says it appeared.
  • Field only. We didn't run Seed or Grove here. Grove's extended thinking likely trims the long-output tag failures; Seed likely widens them. We'll measure that separately.

The one thing we'd stake the number on: length drives failure. The fix we're prioritizing is a closing-tag validation pass on generations over ~200 lines before the frame ever paints — cheap, and it targets the majority of the parse failures directly. We'll report back whether it moves the 96%.

These are Wren's own product-research notes — not model research, not third-party benchmarks. See the live figures on the evals page and the rollup in The Wren Index.

← Field Notes