Clickable Elements: How Manifest Extracts Every Clickable Element From a Webpage

What Manifest's clickable elements endpoint does

Manifest is an API that turns any webpage into a structured manifest AI agents can actually understand and act on. One of the core things it extracts is clickable elements — every button, link, form input, toggle, and dropdown on a page, returned as clean, structured JSON.

Send Manifest a URL like https://example.com/checkout, and among the rest of the page manifest, you get something like:

json

{
  "page": "Checkout",
  "elements": [
    { "id": "submit-order", "label": "Place order", "type": "button", "required": true },
    { "id": "promo-code", "label": "Apply promo", "type": "text_input", "field": "coupon" },
    { "id": "shipping-method", "label": "Shipping method", "type": "select", "required": true }
  ]
}

No pixel coordinates. No guessing which <div> is secretly a button. Just a literal, structured answer to the question "what can I click on this page, and what happens if I do?" — as one part of the full manifest Manifest returns for a page.

Why extracting clickable elements is a real problem

If you've ever tried to get an AI agent to reliably interact with a website, you've run into one of two approaches, and both are worse than they should be.

Vision-based scraping takes a screenshot, asks a vision model to identify buttons and fields, then clicks on pixel coordinates. It's slow, it's expensive to run at scale, and it breaks the moment a site ships a redesign, changes a font size, or loads slightly differently on a slower connection.

DOM scraping parses the raw HTML and JavaScript and tries to infer what's interactive from tag names, class names, and event listeners. It's noisy — you get cookie banners, hidden nav menus, and tracking scripts mixed in with the two buttons that actually matter — and it gives you no semantic signal about what an element means or when it's safe to use.

Both approaches force an agent to solve a layout-understanding problem before it can even start the real task. That's the gap Manifest is built to close: it extracts every clickable element from a webpage once, as part of a full page manifest, so an agent never has to re-solve "what is this page even made of" from scratch.

How the extraction works

Under the hood, Manifest loads the page in a headless browser and reads its accessibility tree — the same structured representation screen readers use to describe a page to someone who can't see it. ARIA roles, aria-label attributes, semantic HTML tags like <button> and <a>, and form field metadata all carry real signal about what's interactive and what it's called. Manifest extracts that signal, filters out non-interactive noise (decorative elements, hidden nodes, tracking pixels), and normalizes it into a consistent JSON shape regardless of how the underlying site was built.

The output isn't a DOM dump. It's a semantic manifest: each clickable element gets an id, a human-readable label, a type (button, link, text input, select, checkbox, and so on), and — where the page's structure makes it inferable — whether it's required and what it depends on.

What's in the response

For every clickable element, Manifest's response includes:

• Label — the visible or accessible text for the element ("Place order", "See Plans")

• Type — button, link, text input, select, checkbox, toggle, and similar categories

• Identifier — a stable reference an agent can use to target the element in a follow-up action

• Required state — whether the element must be interacted with to complete a flow, when that's determinable from the page

• Field metadata — for inputs, what data the field expects (email, coupon code, quantity, and so on)

That's the difference between "here's a screenshot, good luck" and "here's exactly what's clickable, in a format you can act on immediately" — and it's returned alongside the rest of the page manifest (page type, state, and available actions), not as a standalone product.

Who needs this

The clearest use case is AI agents that need to interact with real websites — filling out forms, completing checkouts, navigating multi-step flows — without a human watching over their shoulder or a custom scraper written for every single target site. An agent that receives Manifest's clickable-elements output can plan its next action directly, instead of burning tokens and time interpreting a screenshot or parsing thousands of lines of HTML.

It's also useful for QA and testing tools that need to enumerate every interactive element on a page without manually maintaining selectors, and for accessibility tooling that wants a structured view of what a page exposes as interactive.

A quick example

Say you point Manifest at a marketing site. The manifest it returns might tell you:

• This is a marketing page, not a checkout or dashboard

• There's a button labeled "See Plans"

• Navigation links exist to five other pages

That's enough for an agent — or a script — to decide what to do next, without ever taking a screenshot or parsing a single <div>.

Frequently asked questions

What does "clickable elements" mean in this context? Any element on a page that a user could interact with to trigger a change: buttons, links, form inputs, dropdowns, checkboxes, and toggles. It excludes purely decorative or non-interactive content like paragraph text or images that aren't wrapped in a link.

Is there a separate "Clickable Elements API"? No — clickable element extraction is a feature of the Manifest API, not a standalone product. When you call Manifest on a page, the elements it can interact with are one part of the structured manifest returned, alongside page type, state, and available actions.

Does Manifest return a screenshot or just data? Just data. Manifest returns structured JSON — no image, no pixel coordinates. That's what makes it usable by agents and scripts, not just by something that can "see" a picture.

Does it work on any website? It works on any page a headless browser can load, since it's built on standard web accessibility signals (ARIA roles, semantic HTML, form metadata) rather than a site-specific integration. Extraction quality depends on how well the underlying page uses accessible, semantic markup — sites with poor accessibility practices will return sparser results.

How is this different from scraping HTML directly? Raw HTML scraping returns everything on the page — nav chrome, cookie banners, tracking scripts, and all — and leaves it up to you to infer what's actually interactive. Manifest does that filtering and interpretation for you, returning only what's clickable, labeled in plain language, with no layout-parsing required.

Try it

If you're building an agent that needs to act on real webpages instead of just reading them, Manifest's clickable-elements extraction is the layer that tells it what's actually possible on the page — before it clicks anything.