Scorecard

41 assistants, 15 dimensions, one scale.

Benchmark v0.215 tasks5 of 41 testedlast test Sep 8, 2026updated Sep 8, 2026

Scored 1–10 after real use.

Assistant
Muse7s9.2979101010
Instinct20s8.6810999888
Grok Bot10s7.887887888
Poke11s6.438757858
Town12s3.3622
personal/local
Orchid
Vellum
Folk
Wajo / Fo
Ollie
szn
Tomo
tinyNature
Catch
Caddy
Pally
Joshu
Asmi
OpenInstinct23s
Buoy
Lucas
Mana
Asaply
Waldo
HireAlpha
Boba
Sircle
Moves

9 6 2 score11s median reply not testedN/A doesn't applyHow scoring works

Latest

New tests and quotes.

InstinctTestedSep 8, 2026
10Travel booking · Pass · observed · Books flights and hotels by logging into my own accounts rather than a single aggregator, so it has the full supply of the big sites; end-to-end booking with check-in and boarding pass, seamless.
MuseTestedSep 8, 2026
9Carrying out an online task · Partial · test · No rooms under $250 near the Loop that weekend (Bears game and an expo in town); returned four fully refundable options with totals and cancellation dates and asked whether to raise the budget, move farther out, or shift dates.
MuseTestedSep 8, 2026
9Recommendation quality · Pass · test · Three real options within walking distance with reasons, honest that one was over budget and one borderline on the no-chain rule, a clear pick; booked the table once OpenTable was connected.
MuseTestedSep 8, 2026
10Responding to emails · Pass · observed · Found the thread, read the calendar, drafted a reply-all to both people with two open slots in my voice, asked before sending, sent on command.
Grok BotOtherSep 8, 2026

I just booked a cold email call without lifting a finger Initial email sent with ListKit My Grokbot replied, handled David's objection, pushed for a time David verbally agreed to 2PM Thursday, Grokbot booked him in on our calendar Completely passive callflow https://t.co/nMPzhwexNx

Grok BotOtherSep 8, 2026

Grok Bot, day 4 of 10. Before, I could watch about four agent sessions before turning into a message router. The bottleneck has moved. Now the constraint is how many jobs I can define clearly enough to hand off. That's a better bottleneck. I'm still learning to handle it.

@davemerwinOpen on X
InstinctUse caseSep 8, 2026

Instinct stores my passwords in a vault so it can log into my accounts for me. works great, but giving an ai standing access to your accounts is a real ask. what if vault entries had an expiry - "my amazon, today only" - and when it lapses, it signs itself out? @noahrshinn feature idea from a happy user

InstinctUse caseSep 8, 2026

I’ve been stress-testing Instinct with real-life tasks — here’s what worked, what didn’t, and where I still don’t trust agents Hey everyone — first post here 👋 I’ve been experimenting with Instinct and decided to give it a few tasks that go beyond the usual “summarize this” or “write me an email” use cases. Here’s what I tried: 1. Asked it to order snacks for me across different apps. 2. Asked it to monitor flight prices within a date range, identify good deals, and remind me if prices looked like they were rising or dropped. It was checking roughly 3 times a day. 3. Asked it to check whether my parents had gone for their post-dinner walk. 4. Asked it to research a few tech startups and give me concise summary notes. Overall, I was actually pretty impressed with how well it handled these. In fact I asked it how it can ping my parents instead of just monitoring and it was able to help me with the steps as well. But I also ran into a few pain points: 1. Speed It’s still noticeably slow in some workflows. I’m sure this is something the team is working on but there is huge gap between steps of a task. 2. Browser/site access + human-in-the-loop It still couldn’t access every site/service I wanted it to or did exist. There were also several points where a human had to step in, particularly when it needed to otps or interact with things that weren’t accessible to the agent. 3. Payments are still a big trust issue for me This is probably my biggest concern. I’m comfortable letting an agent find something, compare prices, or even prepare an order. But I’m still not comfortable giving it my card details and letting it freely make payments. Curious how others are using Instinct (or similar agents I can try next). \* What real-world tasks have you trusted an agent with? \* How are you using it currently or want to use it where it is lacking? \* Have you found any good workarounds for payments? \* And most importantly — what tasks would you trust an agent t

u/Sad-Locksmith5980Open on Reddit