Our content lead, Bea, has a particular sigh. It's short, it's through the nose, and it means "the AI has produced eight paragraphs of confident nonsense again." I heard it maybe forty times a week for most of 2024.
Six weeks after we moved her team onto Claude Sonnet 4.5, I heard it twice. That's not a benchmark, and Bea would be embarrassed to be a metric, but it's the honest headline of this teardown.
What we tested, and how
Three teams, six weeks, real work only — no synthetic tasks:
- Content team (4 people): long-form drafts, editing, repurposing 3,000-word reports into briefings.
- Support team (6 people): reply drafting and ticket summarisation, roughly 900 tickets/week.
- Engineering (2 people): tool-calling inside our internal agent stack, plus code review assistance.
We kept our previous setup running in parallel for the first fortnight and had people rate outputs blind where practical. Not rigorous science. Better than vibes.
Key specs
| Spec | Detail | What it meant for us |
|---|---|---|
| Context window | 200K tokens (1M available in beta tiers) | We stopped chunking most documents. Also stopped watching our bill, briefly, which was a mistake. |
| Tool use | Parallel tool calls, structured outputs | Biggest practical win. Our agent's tool-selection error rate fell from about 9% to under 2%. |
| Extended thinking | Optional, budgeted per request | Great for analysis tasks. Overkill for reply drafting — we turn it off there. |
| Vision | Images, screenshots, charts, PDFs | Support team pastes error screenshots straight in. Small thing, saves real minutes. |
| Data handling | API inputs not used for training by default; enterprise controls available | Passed our legal review with less argument than expected. |
Pricing tiers (as we experienced them)
| Tier | Price | Who it suits |
|---|---|---|
| Free (claude.ai) | £0 | Trying it out. Usage caps hit fast on real work. |
| Pro | ~$20 / user / month | Individual knowledge workers. Our content team lives here. |
| Team | ~$25–30 / user / month, 5 seat minimum | Shared projects, admin controls. What we settled on for support. |
| Enterprise | Custom (we were quoted mid-five-figures annually) | SSO, audit logs, expanded context, data residency talk. |
| API | ~$3 per million input tokens, ~$15 per million output | Anything programmatic. Prompt caching cut our real cost by around 55%. |
Prices as of our last invoice — check Anthropic's pricing page, it moves.
What we loved
- Long documents. It holds the thread of a 40-page report far better than anything else we've run. Summaries stayed accurate at page 38.
- Tool reliability. The quiet hero. Fewer malformed calls means fewer retries means a smaller bill.
- It says "I don't know". Not always, but noticeably more often. Bea's sigh count is the evidence.
- Tone control. Give it three examples of house style and it holds them across a whole batch.
- Prompt caching. Easy to implement, immediate cost drop. Do it on day one.
What we'd warn you about
- The context window is a budget hazard. "Just paste the whole thing in" is a habit that cost us £310 in one enthusiastic fortnight.
- Extended thinking is slow. Lovely results, but not for anything a human is waiting on.
- It over-explains. Without instruction you get preamble. "Answer in under 80 words, no preamble" fixes it.
- Rate limits on lower tiers bite during batch jobs. Plan for backoff or upgrade.
- Vendor concentration. Everything we built assumed one provider. We've since added a fallback route — see the post-mortem.
What it actually cost us
Six weeks, twelve people, mixed seats and API:
- Seats: £1,180
- API (support automation + agent stack): £742
- The enthusiastic-pasting incident: £310 of that, honestly
- Total: £1,922
Against that: the support team's median first-response time went from 4h10m to 1h50m, and the content team shipped 31% more published pieces without anyone working later. We'd pay it again.
Crew verdict — 8.7 / 10
If your work involves reading long things and writing careful things, this is currently the easiest recommendation we can make. The tool-use reliability makes it the default choice for agent builds too.
Skip it if: you're doing high-volume, narrow classification (fine-tune a small model instead — see the RAG vs fine-tune thread), or if you need sub-500ms responses on every call.
How we'd set it up on day one
- Turn on prompt caching for anything with a stable system prompt. Non-negotiable.
- Set a monthly spend cap and a daily usage alert at 130% of normal.
- Write one house-style prompt with three real examples, store it centrally, don't let ten people fork it.
- Disable extended thinking on anything customer-facing and latency-sensitive.
- Log every request with a project tag so you can attribute cost later. Gwen will thank you.
Official links
- Claude product page
- Anthropic developer documentation
- Current pricing
- Trust & security centre — what our legal team wanted
Crew replies
Confirming the tool-use point from the engineering side. Our agent retry rate dropped enough that the migration paid for itself in reduced token waste inside a month. That's a weird sentence but it's true.
Please, everyone, tag your API requests by project from the start. I spent an entire afternoon trying to work out who spent £310 on pasting PDFs. (It was three different people, all being helpful.)
Compare it with
We've torn down two other things you'll probably be weighing this against:
OpenAI Agents SDK → n8n teardown →