The short answer
Pick LangSmith if you build and deploy agents, especially with LangChain or LangGraph, and need tracing, evaluation and deployment infrastructure. Pick AgentPilots if you run many existing agents across tools like Grok Bot, Claude Code, Cursor and Codex and want to see live who is working and who needs you.
What each tool is
LangSmith, from LangChain, covers observability ("see exactly what your agents are doing"), evaluation and agent infrastructure/deployment. It is offered as managed cloud, bring-your-own-cloud and self-hosted (langchain.com/langsmith, pricing).
AgentPilots is a lightweight Windows desktop app. Agents call an MCP connector with three tools to report their state and milestones, and the app shows working, needs-approval and idle indicators, live activity cards, a graph view and per-client connection snippets.
Feature comparison
| AgentPilots | LangSmith | |
|---|---|---|
| Live working / needs-approval / idle status | Yes | No |
| No SDK, proxy or code changes | Yes | No |
| Grok Bot, Claude Code, Cursor, Codex in one view | Yes | No |
| Full trace capture & replay | No | Yes |
| Evals & prompt management | No | Yes |
| Token & cost tracking | Soon | Yes |
| Open source | No | No |
| Lightweight desktop app | Yes | No |
See all details
| Capability | AgentPilots | LangSmith |
|---|---|---|
| Primary job | Live operations view: which agents are working, need approval or are idle | Agent engineering: observability, evaluation, deployment |
| Built for | People who run many agents (operators, founders, agencies) | Developers and teams building agents |
| How agents connect | MCP connector (3 tools). No SDK, no proxy | SDK / tracing integration (works with and without LangChain) |
| Code changes required | None for MCP-capable clients: add a server and a short instruction | Yes: add tracing to your application |
| Live working / needs-approval / idle indicators | Yes | Not an operator status board |
| Live activity feed of milestones | Yes | Trace views and dashboards |
| Full trace and span capture | No | Yes |
| Evals, datasets, prompt management | No | Yes |
| Token and cost tracking | Not yet (on the roadmap) | Yes (token and cost in traces) |
| Grok Bot, Claude Code, Cursor, Codex agents in one view | Yes (Claude Desktop via mcp-remote; ChatGPT not yet) | Instruments code you own |
| Form factor | Lightweight Windows desktop app | Web app (cloud, BYOC or self-hosted) |
| Open source | No | No (platform); LangChain libraries are open source |
| Self-hosting | Not applicable (local desktop app) | Yes (managed, BYOC or self-hosted; contact sales) |
LangSmith details are from its public website and docs as of October 13, 2026: langchain.com/langsmith, langchain.com/pricing. Tell us if something has changed and we'll fix it.
Who should pick which
Choose LangSmith if
- You build agents with LangChain or LangGraph and want first-party tooling.
- You need evaluations and experiments as part of development.
- You want managed deployment for your agents.
Choose AgentPilots if
- Your agents run in tools you don't control the code of (Grok Bot, Cursor, Claude Code, Codex).
- You need a live "who needs me?" view more than trace analysis.
- You prefer a small desktop app over another web platform.
Pricing summary
| Plan | Published price (checked October 13, 2026) |
|---|---|
| Developer | $0 per seat / month, 1 seat, up to 5k base traces / month, then pay as you go |
| Plus | $39 per seat / month |
| Enterprise | Custom |
Source: https://www.langchain.com/pricing. Vendor prices change, so always check the source. AgentPilots is free during the beta; paid tiers have not been announced.
Can you use both?
Yes. A LangGraph agent can be traced in LangSmith and also call AgentPilots' report_status through an MCP client, so it shows up live next to your other agents. The two tools answer different questions.
FAQ
Is AgentPilots a LangSmith alternative?
For live operational status, yes. For tracing, evaluation and deployment, no: LangSmith covers those and AgentPilots does not.
Does AgentPilots work with LangChain or LangGraph agents?
Any agent that can call an MCP tool can report to AgentPilots. Tested setup snippets today cover Grok Bot, Claude Code, Cursor, Codex and Claude Desktop (via mcp-remote).
How much does LangSmith cost?
As of Oct 13, 2026, langchain.com/pricing lists Developer at $0 per seat (1 seat, up to 5k base traces/month, then pay as you go), Plus at $39 per seat per month, and custom Enterprise pricing. The page also describes usage-based LangChain Standard Units for some products; check it for current details.
Can I self-host AgentPilots?
There is nothing to host: AgentPilots is a local Windows desktop app. LangSmith offers self-hosting for teams with data residency needs.
Does AgentPilots support ChatGPT?
Not yet. ChatGPT needs OAuth sign-in on the MCP server, which AgentPilots does not support today.
ready for takeoff?
AgentPilots is in private beta. Join the waitlist and we'll email you a download link when it's ready.