Skyvern 2.0 just posted 85.8% on WebVoyager. Someone shared it on Hacker News. The comments blew up.

And the honest reaction most people had — including me — was: okay, but what does that number actually mean?

The leaderboard nobody asked for (but everyone needed)

There’s now a public leaderboard tracking AI browser agent performance. This is genuinely new. Six months ago you had to just trust whatever marketing copy a company put on their landing page. “Best-in-class accuracy.” Sure. Verified by whom?

The leaderboard changes that. It’s a shared benchmark so you can at least compare apples to apples. WebVoyager is the main one being used — it’s a set of real-world web tasks across different sites. Fill out this form. Find this product. Book this flight. Stuff people actually do.

Skyvern hit 85.8%.

That sounds high. Is it high?

What 85.8% looks like in practice

Here’s a thought experiment. Imagine hiring an assistant who fails roughly 1 in 7 tasks.

You’d fire them, right?

But wait. That’s not quite the right frame. These are hard tasks, on live websites, with no setup, no walkthroughs, nothing. The agent is dropped into the wild and told to figure it out. The fact that anything clears 80% is kind of remarkable.

WebVoyager tasks aren’t “click the login button.” They’re more like “find me the cheapest roundtrip flight from NYC to London in March with at most one stop.” On the actual web. No sandbox. Real sites that change layout constantly.

So 85.8% is impressive. But it’s also… a benchmark. And benchmarks are weird.

Benchmarks lie a little. Not intentionally.

I’m not calling anyone dishonest here. But every benchmark measures a specific slice of reality. WebVoyager has 643 tasks across 15 websites. That’s enough to be meaningful. It’s not enough to be complete.

Think about what’s not in WebVoyager. Your company’s internal HR portal. That one ancient government form you have to submit quarterly. The client’s CRM that hasn’t been updated since 2018. The random SaaS tool your team adopted last month.

The long tail of browser work is enormous. And weird. And highly personal.

A model that aces WebVoyager might completely choke on filling out a specific procurement form with three dependent dropdown menus that load via obscure JavaScript events. Or it might handle it perfectly. The benchmark doesn’t tell you.

This isn’t a criticism of the benchmark. It’s just reality. Anyone selling you certainty based on benchmark scores is oversimplifying.

Why this is still worth paying attention to

Okay, but the leaderboard still matters. Here’s why.

Before it existed, you couldn’t even have this conversation. You couldn’t compare Skyvern to anything. You couldn’t see progress over time. You couldn’t hold anyone accountable to claims.

Now you can. The WebVoyager score gives you a starting point. It’s an anchor. If a new agent launches claiming to be the best and they score 62% on WebVoyager, that tells you something real — even if it’s not the whole story.

And Skyvern’s jump specifically is worth noting. These aren’t small incremental gains. When an agent goes from “sometimes works” to 85.8% on a hard benchmark, the underlying model quality has meaningfully shifted. That affects everyone building in this space, not just Skyvern users.

It raises the floor for what’s acceptable.

The part benchmarks can’t measure

Here’s what I keep coming back to.

Most AI browser agent benchmarks measure whether the agent completes the task. Did it book the flight? Yes or no. Pass or fail.

But real users have more complicated needs.

Did the agent show me what it was doing? Did it ask before clicking “confirm purchase”? Did it store my login credentials somewhere I didn’t agree to? Did my boss’s email drafts just get sent to some server in order to process them?

None of this shows up in a benchmark score.

This is where the gap between benchmark performance and daily usefulness gets interesting. A system that scores 90% but silently processes your data through someone else’s servers might be technically impressive and practically wrong for your situation.

Not everyone cares about this, to be clear. If you’re automating public research on company websites, privacy is maybe not your top concern. But if you’re drafting emails, filling HR forms, logging into work tools — the data exposure question is pretty legitimate.

Where bring-your-own-key fits in

There’s a whole category of browser agents that don’t score on leaderboards because they’re not built as standalone products. They’re built to sit inside your browser and use your API key with Claude, GPT, Gemini, whatever you prefer.

Dassi works this way. Lives in the side panel. Sees what you see. Uses your own LLM key. Nothing gets routed through our servers. Free.

The tradeoff is real: a purpose-built, fully hosted agent that’s been optimized and tuned specifically for WebVoyager-style tasks will probably outperform a general browser assistant on that benchmark. Skyvern is doing serious work on their infrastructure and it shows.

But “outperform on a benchmark” and “better for your day-to-day browser work” aren’t always the same thing.

If you’re doing repetitive tasks in your own browser context — email drafts, research, form fills, reading through long documents while you work — a tool that sees exactly what you see and uses a model you trust, with your own API key, has real advantages. Even if it never appears on a leaderboard.

A tangent about why benchmarks became a thing

Slightly off-topic but I find this genuinely interesting.

The benchmark culture in AI comes from academia. NLP researchers needed a way to compare models before large language models existed. GLUE, SuperGLUE, SQuAD — these were created so researchers could make apples-to-apples comparisons and publish papers claiming improvement.

The problem is that once a benchmark becomes famous, everyone optimizes for it. The metric stops being a good proxy for the thing you care about. Goodhart’s Law, basically.

WebVoyager is relatively new, so we’re not there yet. But watch the leaderboard over the next year. You’ll see scores converge near the top as every major player specifically trains on tasks that look like WebVoyager. And then the benchmark will need to get harder.

This isn’t bad. It’s just how it works. The target moves, which pushes capability forward, which benefits users eventually.

So should you care about the leaderboard?

Yes, with caveats.

High benchmark scores correlate with better underlying capability. Skyvern at 85.8% is probably doing real things right. The gap between 62% and 85% likely reflects genuine differences in how the system handles complex multi-step tasks.

But don’t pick an AI browser agent based on a score alone.

Ask: Does it work on the specific sites and tasks I actually care about? What happens when it fails — does it tell me, or does it silently do the wrong thing? What data does it see, and where does that data go? What does it cost, and is that cost per-task or flat?

The leaderboard is a filter, not a verdict.

If you’re curious what the day-to-day experience of a browser agent actually looks like — before ever worrying about benchmarks — our post on AI Browser Agents: The 2026 Guide walks through the basics. And if you’re wondering whether you even need one, the 7 tasks you still do manually post is a decent gut check.

The scores are getting better. The category is maturing. And the conversation about what these tools actually do — and what they do with your data — is only getting more important.

Good time to be paying attention.