Why aria_snapshot() Isn't Enough for Agents That Need to Submit Forms

If you've built a browser agent with Playwright, you've probably leaned on aria_snapshot() to give your LLM a readable view of the page. It's a genuinely good tool — it collapses a noisy DOM into a clean accessibility tree, and for read-and-click tasks it usually works fine.

Where it falls apart is forms. Not simple ones — a login form with two fields is trivial for any approach. The ones that break agents are the ones with order: a shipping form where the state dropdown only populates after country is selected, a checkout flow where the submit button stays disabled until three unrelated fields pass validation, a signup form where an email-verification step gates the rest.

aria_snapshot() doesn't know any of that. It tells you an element exists, its role, and its accessible name. It doesn't tell you what has to happen before that element becomes interactable, or what breaks if you fill it out of order.

What the snapshot actually gives you

A typical aria_snapshot() output for a form field looks something like:

- textbox "State" [disabled]

That's useful — you know there's a State field, and you know it's currently disabled. What you don't get is why it's disabled, or what action would change that. An LLM agent seeing this has two options: guess, or try clicking around until something works. Both are expensive. Guessing wrong on a checkout form usually means restarting the whole flow. Trying-until-it-works burns tokens and time on every single run, and it doesn't get more reliable the second time you hit the same site.

This is the same failure mode whether you're using raw Playwright, a wrapped browser-use agent, or any tool that treats the accessibility tree as the ground truth. The tree is a snapshot of state. It was never designed to encode sequence.

The gap gets worse at scale

For a one-off script against a site you already understand, you can hardcode the sequence yourself. That doesn't scale to an agent that needs to handle sites it's never seen before, or to a team running the same extraction across hundreds of target domains.

Two specific failure patterns show up constantly once you're past toy examples:

Cross-field dependencies. Field B's validity depends on field A being filled correctly first, but nothing in the accessibility tree signals that relationship. The agent has to discover it empirically, per page, per run.

Resolved locators that don't survive re-renders. Even once an agent figures out the right sequence, the selectors it used to get there are often fragile — auto-generated class names, positional CSS paths — and break the next time the page ships a minor frontend update.

What we do differently

This is the specific problem Manifest's requires field is built to solve. When Manifest extracts a page, every actionable element comes back with a requires array listing the other actions that need to happen first — not as a heuristic the agent has to infer, but as an explicit, executable dependency graph:

json

{
  "action": "submit_shipping_form",
  "selector": "button[type='submit']",
  "requires": ["fill_country", "fill_state", "fill_zip"]
}

Alongside that, every locator Manifest returns is resolved against the live DOM at extraction time — CSS selector, ARIA role, and accessible name together — rather than a single brittle string. If the page structure shifts slightly, the agent still has enough signal to find the right element.

The result is that an agent doesn't have to rediscover form logic on every run. It gets the sequence up front, as data, the same way it gets the elements themselves.

Where this fits

aria_snapshot() isn't wrong — it's solving a narrower problem than "make this page fully actionable for an autonomous agent." Manifest sits a layer above it: same underlying idea of exposing what's on a page in a structured way, but extended to cover what order things need to happen in and how to find them reliably after the DOM changes.

If you're building an agent that only reads pages, you probably don't need this. If you're building one that has to get through a real form — checkout, signup, application, anything with validation logic — the dependency graph is the piece that's usually missing.

Manifest turns any webpage into a structured JSON action manifest — every clickable, fillable, and submittable element, with resolved locators and dependency data included. Try the live demo or read the docs.