AI Agent Developer

A TypeScript-first path to building AI agents that call tools, search private data, connect through MCP and run safely in production. It covers LLM basics, context engineering, the agent loop, retrieval, multi-agent workflows, evals and security, with Python where the ecosystem needs it.

9 stages62 topicsAbout 7-9 months at 10 hours a week

Before you start: For JavaScript and MERN developers who can already build and deploy a small web app with a REST API. You should be comfortable with Node.js, npm and Git; no machine learning background is needed.
Where it gets you: You can design, build, evaluate and deploy tool-using agents with retrieval, MCP integrations and durable workflows, measure their quality and cost, and defend them against prompt injection and misuse.

0 of 62 topics done

Tick a topic once you can explain it and have used it in a project. Progress is saved in this browser.

Core
Recommended
Optional
Done

1
Foundations for Agent Work
About 3-4 weeks0/7
Write typed, async TypeScript that calls HTTP APIs, validates JSON and handles failures cleanly.
Build: Typed API Client with Retries
Build a TypeScript CLI that calls a public JSON API, validates every response with Zod, retries 429 and 5xx errors with backoff, and cancels cleanly on Ctrl+C. Add a GitHub Actions job that runs the type-check and tests.
2
LLM Fundamentals
About 2-3 weeks0/7
Call frontier models from code, stream their output, and choose a model based on quality, cost and latency.
Build: Streaming Chat with a Model Switcher
Build a Next.js or Express chat app that streams replies from two providers, lets the user switch models mid-conversation, and shows tokens, cost and time to first token for every message.
3
Prompt and Context Engineering
About 2-3 weeks0/6
Write prompts that produce reliable, schema-valid output and control what goes into the context window on every call.
Build: Document Extractor
Build a service that turns PDF or image invoices into validated JSON (vendor, line items, totals) using a strict schema and a cached system prompt. Log cost and cache hits for every document.
4
Tool Use and the Agent Loop
About 4-5 weeks0/7
Build a single agent that calls well-designed tools in a loop, stops safely, recovers from errors and asks a human before risky actions.
Build: Issue Triage Agent
Build an agent with GitHub API tools that reads new issues, searches the codebase, suggests labels and drafts a reply. Give it a step limit, a cost cap and a human approval step before anything is posted.
5
Knowledge, Retrieval and Memory
About 4-5 weeks0/7
Give an agent accurate access to private data through search tools, cite its sources and remember what matters across sessions.
Build: Docs Assistant with Citations
Index a real documentation site into pgvector or MongoDB Vector Search, expose hybrid search with reranking as an agent tool, and answer questions with cited passages. Include a 30-question test set that measures recall@5.
6
MCP and Integrations
About 3-4 weeks0/7
Build, secure and connect MCP servers so any compatible agent can use your tools and data.
Build: MCP Server for Your Own App
Wrap the API of an existing MERN app (tasks, orders or notes) as a remote MCP server with OAuth, read-only resources and a few write tools. Connect it to two different MCP clients and test it with the Inspector.
7
Multi-agent Systems and Durable Workflows
About 4-5 weeks0/7
Split large tasks across coordinated agents and run them as durable jobs that survive crashes and wait for people.
Build: Parallel Research Pipeline
Build a research agent whose lead agent splits a question into subtasks, runs three to five subagents in parallel as durable steps, survives a server restart mid-run, and emails a cited report when it finishes.
8
Evals and Observability
About 3-4 weeks0/7
Measure agent quality with repeatable evals, catch regressions in CI and trace every production run down to each tool call.
Build: Eval Suite in CI
Add a 50-case eval suite to your triage or docs agent with code checks and a calibrated LLM judge, and run it in GitHub Actions on every pull request. Export traces from each run to an OpenTelemetry backend.
9
Safety, Security and Production
About 4-5 weeks0/7
Ship agents that resist prompt injection, run with least privilege, stay within budget and keep working when a provider fails.
Build: Production Launch
Ship one of your agents publicly with per-user auth, sandboxed code execution, a prompt injection test suite, provider fallback, per-user quotas and a cost dashboard. Then write a one-page incident runbook.
Where it gets you
You can design, build, evaluate and deploy tool-using agents with retrieval, MCP integrations and durable workflows, measure their quality and cost, and defend them against prompt injection and misuse.

Core must know for the roleRecommended expected in most jobsOptional for specific teams or later

1
Foundations for Agent Work
About 3-4 weeks
Write typed, async TypeScript that calls HTTP APIs, validates JSON and handles failures cleanly.
- TypeScript Essentials
  Core
  Agent code moves untyped model output through typed application code, so strict TypeScript catches most integration bugs before users see them.
  - Strict mode and type narrowing
  - Unions and discriminated unions
  - Generics for reusable helpers
  - Zod schemas with inferred types
  - ES modules and package scripts
  - Node.js runtime basics
  - TypeScript Handbook
- Async Code and Streams
  Core
  Model calls are slow network requests that often stream, so you need to be fluent with promises, cancellation and async iteration.
  - async/await and error handling
  - Promise.all with concurrency limits
  - AbortController for timeouts
  - Async iterators with for await
  - Web Streams and ReadableStream
  - Retries with exponential backoff
  - MDN: Using promises
  - MDN: Streams API
- HTTP APIs and Server-Sent Events
  Core
  Every provider serves models over HTTP and streams with server-sent events, so this is the plumbing under every SDK you will use.
  - Methods, status codes and headers
  - Keeping API keys server-side
  - Server-sent events for streaming
  - Webhooks and signature checks
  - Handling 429 and retry-after
  - Routes in Express or Next.js
  - MDN: Using server-sent events
- JSON and JSON Schema
  Core
  Tool definitions and structured outputs are written in JSON Schema, so reading and writing schemas becomes a daily task.
  - Types, properties and required
  - Enums, formats and nested objects
  - Descriptions that guide the model
  - Converting Zod to JSON Schema
  - Validating untrusted input
  - JSON Schema: Getting started
  - Zod: JSON Schema
- Git, Secrets and Project Setup
  Core
  Agent projects hold API keys and change prompts often, so clean version control and secret handling matter from day one.
  - Branches and pull requests
  - .env files and secret managers
  - Never commit or log API keys
  - Pin SDK versions with lockfiles
  - Basic CI with GitHub Actions
  - Git documentation
- Python as a Second Language
  Optional
  Much of the research, eval and data tooling is Python-first, so basic Python lets you read examples and use those tools when needed.
  - Virtual environments with venv or uv
  - asyncio basics
  - Pydantic models for validation
  - Notebooks for quick experiments
  - Porting Python examples to TypeScript
  - Python docs: asyncio
- Math Intuition
  Optional
  You do not need calculus to build agents, but a feel for vectors and probability makes embeddings, sampling and eval metrics easier to reason about.
  - Vectors and dot products
  - Cosine similarity
  - Probability over next tokens
  - Precision, recall and averages
Build: Typed API Client with Retries
Build a TypeScript CLI that calls a public JSON API, validates every response with Zod, retries 429 and 5xx errors with backoff, and cancels cleanly on Ctrl+C. Add a GitHub Actions job that runs the type-check and tests.
2
LLM Fundamentals
About 2-3 weeks
Call frontier models from code, stream their output, and choose a model based on quality, cost and latency.
- How LLMs Generate Text
  Core
  Knowing that models predict tokens inside a fixed context window explains most of their limits, costs and failure modes.
  - Tokens and tokenizers
  - Next-token prediction
  - Context window limits
  - Input vs output token pricing
  - Knowledge cutoffs and hallucination
  - Reasoning or thinking modes
  - Claude docs: Context windows
- The Messages API
  Core
  Every provider uses a similar request of roles and messages, and every agent is built by calling it in a loop.
  - System, user and assistant roles
  - Resending history each turn
  - Stop and finish reasons
  - Usage fields for token counts
  - Error types and retry rules
  - Official TypeScript and Python SDKs
  - Claude docs: Messages API
  - OpenAI docs: Text generation
- Generation Settings
  Core
  Settings like max tokens, temperature and reasoning effort change quality, cost and speed, and newer reasoning models ignore some of them.
  - Max output tokens
  - Temperature and top_p
  - Stop sequences
  - Reasoning effort controls
  - Why outputs vary between runs
  - Claude docs: Effort
- Streaming Responses
  Core
  Users expect to see output as it is generated, and agents stream text, tool calls and progress events the same way.
  - Server-sent event chunks
  - Text deltas and final message
  - Streaming tool call arguments
  - Rendering streams in React
  - Cancelling a stream mid-response
  - Claude docs: Streaming
  - AI SDK: Chatbot UI
- Choosing Models
  Core
  Model choice is the biggest lever on cost and latency, and the right answer shifts every time providers ship new releases.
  - Frontier vs small fast models
  - Anthropic, OpenAI, Google, open-weight
  - Price per million tokens
  - Time to first token
  - Batch APIs for offline jobs
  - Test on your own tasks
  - Claude docs: Choosing a model
  - Gemini API: Models
- Multimodal Inputs
  Recommended
  Agents increasingly read screenshots, PDFs and scans, which changes both what they can do and what each request costs.
  - Images and screenshots as input
  - PDF and document input
  - Files APIs to avoid re-uploads
  - Token cost of images
  - Audio and realtime APIs
  - Claude docs: Vision
  - Claude docs: PDF support
- Provider-agnostic SDKs
  Recommended
  A unified SDK lets you switch providers without rewrites, which matters when prices and model rankings change every few months.
  - AI SDK generateText and streamText
  - Swapping providers through config
  - Provider-specific options
  - AI gateways for routing and billing
  - When to use a native SDK
  - AI SDK: Generating text
  - Vercel docs: AI Gateway
Build: Streaming Chat with a Model Switcher
Build a Next.js or Express chat app that streams replies from two providers, lets the user switch models mid-conversation, and shows tokens, cost and time to first token for every message.
3
Prompt and Context Engineering
About 2-3 weeks
Write prompts that produce reliable, schema-valid output and control what goes into the context window on every call.
- Writing Clear Instructions
  Core
  Frontier models follow instructions closely, so most quality problems come from vague or missing context rather than from the model.
  - State task, audience and format
  - Explain context and the reason why
  - Separate sections with XML tags
  - Few-shot examples of good output
  - Say what to do, not only avoid
  - Keep prompts in version control
  - Claude docs: Prompt engineering
  - Claude docs: Prompting best practices
- Structured Outputs
  Core
  Agents and back ends need data they can parse, and schema-constrained output makes JSON dependable enough to build on.
  - JSON mode vs strict schemas
  - Required fields and enums
  - Zod or Pydantic definitions
  - Handling refusals and truncation
  - Validate before any side effect
  - Claude docs: Structured outputs
  - AI SDK: Structured data
- Context Engineering
  Core
  Long agent runs fill the window with tool output, and deciding what stays, what gets summarized and what is fetched later is now a core skill.
  - What the model sees each turn
  - Quality drops in very long contexts
  - Fetch just in time, not upfront
  - Trim and clear old tool results
  - Compaction and running summaries
  - Notes files for long tasks
  - Claude docs: Compaction
  - Claude docs: Context editing
- Prompt Caching
  Core
  Agents resend the same system prompt and tools on every step, so caching the stable prefix cuts cost and latency by a large margin.
  - Stable prefix first, variable last
  - Cache breakpoints and lifetimes
  - Cache write vs read pricing
  - Measuring cache hit rate
  - Keep tool order deterministic
  - Claude docs: Prompt caching
  - OpenAI docs: Prompt caching
- Token Counting and Budgets
  Recommended
  Counting tokens before a request lets you enforce limits and predict spend instead of finding out from the invoice.
  - Count tokens before sending
  - Set max tokens per call
  - Budgets per user and task
  - Estimate monthly spend
  - Claude docs: Token counting
- Grounding and Hallucination Control
  Recommended
  Models will answer confidently without facts, so you need prompt patterns that tie answers to the sources you supply.
  - Put source documents in context
  - Ask for quotes before answers
  - Let the model say it is unsure
  - Verify claims with a second pass
  - Claude docs: Reduce hallucinations
Build: Document Extractor
Build a service that turns PDF or image invoices into validated JSON (vendor, line items, totals) using a strict schema and a cached system prompt. Log cost and cache hits for every document.
4
Tool Use and the Agent Loop
About 4-5 weeks
Build a single agent that calls well-designed tools in a loop, stops safely, recovers from errors and asks a human before risky actions.
- Tool Calling
  Core
  Tool calling is how a model acts on the world: it requests a function call, your code runs it and sends back the result.
  - Name, description and input schema
  - Model requests, your code executes
  - Returning tool results by ID
  - Parallel tool calls
  - Forcing or disabling tool choice
  - Strict validation of arguments
  - Claude docs: How tool use works
  - OpenAI docs: Function calling
- Designing Good Tools
  Core
  An agent is only as good as its tools, and tool names, descriptions and outputs are effectively part of the prompt.
  - Fewer, task-level tools
  - Clear names and parameter docs
  - Concise, relevant results
  - Errors the model can act on
  - Pagination and truncation
  - Idempotent write operations
  - Claude docs: Define tools
- The Agent Loop
  Core
  An agent is a model calling tools in a loop until the task is done, and the loop controls decide whether it finishes or runs away.
  - Call model, run tools, repeat
  - Stop on a final answer
  - Max steps and turn limits
  - Token and cost budgets
  - Timeouts per tool call
  - Detect repeated identical calls
  - AI SDK: Loop control
  - Claude Agent SDK: Agent loop
- Workflows vs Agents
  Core
  Many tasks work better as fixed code paths with model steps, so learn the simple patterns before reaching for full autonomy.
  - Prompt chaining
  - Routing to specialized prompts
  - Parallel fan-out and voting
  - Evaluator and optimizer loops
  - Pick the simplest pattern that works
  - AI SDK: Workflow patterns
- Error Handling and Recovery
  Core
  Tools fail, time out and return junk, and an agent that sees clear errors can often fix the problem on its own.
  - Return errors as tool results
  - Retry transient failures with backoff
  - Repair or reject bad arguments
  - Fallback when a tool is down
  - Checkpoint progress for resume
  - Claude docs: Handle tool calls
- Human-in-the-loop Approval
  Core
  Agents that write data, send messages or spend money need a pause for human approval, and users trust agents that ask first.
  - Classify tools by risk
  - Require approval for writes
  - Pause, show the action, resume
  - Approval UI inside the chat
  - Audit log of every decision
  - AI SDK: Tool approvals
  - Claude Agent SDK: Approvals and user input
- Agent SDKs and Frameworks
  Recommended
  Once you understand the raw loop, an SDK saves boilerplate and adds streaming, sessions and tracing, so pick one and learn it well.
  - Vercel AI SDK agents
  - Claude Agent SDK built-in tools
  - OpenAI Agents SDK
  - LangGraph for explicit state graphs
  - When to hand-roll the loop
  - AI SDK: Agents overview
  - Claude Agent SDK overview
Build: Issue Triage Agent
Build an agent with GitHub API tools that reads new issues, searches the codebase, suggests labels and drafts a reply. Give it a step limit, a cost cap and a human approval step before anything is posted.
5
Knowledge, Retrieval and Memory
About 4-5 weeks
Give an agent accurate access to private data through search tools, cite its sources and remember what matters across sessions.
- Embeddings
  Core
  Embeddings turn text into vectors so you can search by meaning, which is the base of most retrieval and memory systems.
  - Text to vectors
  - Cosine similarity search
  - Choosing models and dimensions
  - Batch embedding and cost
  - Re-embed when models change
  - AI SDK: Embeddings
  - OpenAI docs: Embeddings
- Ingestion and Chunking
  Core
  Retrieval quality is decided at ingestion time, so how you parse, split and label documents matters more than which vector store you pick.
  - Parse PDF, HTML and Markdown
  - Chunk size and overlap
  - Split by headings and structure
  - Metadata for filters and citations
  - Re-index only changed documents
- Vector Search in Your Database
  Core
  Most teams store vectors next to their existing data, and both Postgres and MongoDB support vector search today.
  - pgvector columns and indexes
  - HNSW vs IVFFlat trade-offs
  - MongoDB Vector Search for MERN
  - Metadata filters in queries
  - Managed vector databases
  - Tenant isolation in every query
  - pgvector on GitHub
  - MongoDB docs: Vector Search
- Hybrid Search and Reranking
  Core
  Pure vector search misses exact terms like IDs and names, so production systems combine keyword and vector results and rerank them.
  - Full-text or BM25 keyword search
  - Reciprocal rank fusion
  - Rerankers for the top results
  - Query rewriting and expansion
  - Tuning top-k per query type
  - AI SDK: Reranking
  - MongoDB docs: Hybrid search
- Agentic Retrieval and Citations
  Core
  Agents now search step by step with tools instead of receiving one fixed batch of chunks, and citations let users check every answer.
  - Search exposed as a tool
  - Multi-step search and refine
  - grep and file search for code
  - Citations to source passages
  - Refuse when sources are missing
  - Claude docs: Citations
  - Claude docs: Search results
- Evaluating Retrieval
  Recommended
  When an answer is wrong you need to know whether retrieval or generation failed, and a small retrieval test set tells you quickly.
  - Recall@k and MRR
  - Question to source test sets
  - Faithfulness to retrieved text
  - Separate retrieval and answer errors
- Short and Long-Term Memory
  Recommended
  Users expect agents to remember preferences and past work, which needs deliberate design for storage, recall, expiry and privacy.
  - History as short-term memory
  - Summaries for long threads
  - Persistent facts per user
  - Memory tools and memory files
  - Expiry and user deletion
  - No memory shared across tenants
  - AI SDK: Memory
  - Claude docs: Memory tool
Build: Docs Assistant with Citations
Index a real documentation site into pgvector or MongoDB Vector Search, expose hybrid search with reranking as an agent tool, and answer questions with cited passages. Include a 30-question test set that measures recall@5.
6
MCP and Integrations
About 3-4 weeks
Build, secure and connect MCP servers so any compatible agent can use your tools and data.
- MCP Architecture
  Core
  MCP is the standard way agents connect to tools and data, and the major model providers, IDEs and agent SDKs all support it.
  - Hosts, clients and servers
  - Tools, resources and prompts
  - stdio and Streamable HTTP transports
  - Stateless requests in the 2026 spec
  - Capability and version discovery
  - MCP docs: Architecture
  - MCP docs: Server concepts
- Building an MCP Server
  Core
  Wrapping your own API as an MCP server makes it usable from many clients without writing a custom integration for each one.
  - Official TypeScript SDK
  - Tools with input and output schemas
  - Resources addressed by URI
  - Reusable prompt templates
  - Testing with MCP Inspector
  - Deterministic tool list order
  - MCP docs: Build a server
  - MCP docs: Inspector
- MCP Auth and Security
  Core
  Remote MCP servers expose real data and actions, so OAuth, scoped tokens and defenses against malicious servers are required.
  - OAuth 2.1 for remote servers
  - Client ID Metadata Documents
  - No token passthrough upstream
  - Confused deputy and SSRF risks
  - Tool poisoning from untrusted servers
  - Allowlist trusted servers
  - MCP docs: Authorization
  - MCP docs: Security best practices
- Connecting MCP to Agents
  Core
  Agents use MCP through SDK clients or provider-side connectors, and large tool sets need search and filtering to stay usable.
  - MCP clients in the AI SDK
  - Provider-hosted MCP connectors
  - Tool search for large tool sets
  - Filter which tools load
  - Timeouts and server errors
  - AI SDK: MCP tools
  - Claude docs: MCP connector
- Agent Skills
  Recommended
  Skills package instructions, scripts and references into folders an agent loads only when needed, and the open format now works across many agent products.
  - SKILL.md with name and description
  - Progressive disclosure of context
  - Bundled scripts and references
  - Skills vs MCP tools
  - Descriptions that trigger correctly
  - Agent Skills: Specification
  - Claude docs: Agent Skills
- MCP Extensions and User Input
  Optional
  The 2026 spec moved advanced features into extensions and changed how servers ask users for input, which matters for interactive and long-running tools.
  - Elicitation via multi round-trip requests
  - Tasks extension for long jobs
  - MCP Apps for interactive UI
  - Deprecated: sampling, roots, logging
  - MCP spec: Key changes
- Browser and Computer Use
  Optional
  When no API exists, agents can drive a browser or desktop from screenshots, at the cost of speed, reliability and a larger attack surface.
  - Screenshot, act, verify loop
  - Browser automation with Playwright
  - Prefer APIs when they exist
  - Isolated VMs for desktop control
  - Higher latency and token cost
  - Claude docs: Computer use
Build: MCP Server for Your Own App
Wrap the API of an existing MERN app (tasks, orders or notes) as a remote MCP server with OAuth, read-only resources and a few write tools. Connect it to two different MCP clients and test it with the Inspector.
7
Multi-agent Systems and Durable Workflows
About 4-5 weeks
Split large tasks across coordinated agents and run them as durable jobs that survive crashes and wait for people.
- Orchestrators and Subagents
  Core
  A lead agent that hands focused subtasks to subagents with their own context can handle broad work that would overflow a single context window.
  - Lead agent plans and delegates
  - Subagents with isolated context
  - Return summaries, not transcripts
  - Run subagents in parallel
  - Cost grows with agent count
  - Know when one agent is enough
  - Claude Agent SDK: Subagents
  - AI SDK: Subagents
- Handoffs and Routing
  Recommended
  Support and ops agents often pass a conversation to a specialist agent, and handoffs need clear context passing and loop guards.
  - Triage agent routes to specialists
  - Context passed on handoff
  - Agents as tools vs handoffs
  - Guard against handoff loops
  - OpenAI Agents SDK: Handoffs
  - OpenAI Agents SDK: Multi-agent
- Durable Execution
  Core
  Agent runs can take minutes or days, so each step must be checkpointed and retried instead of living in one fragile process.
  - Steps with checkpoints and retries
  - Resume after crashes and deploys
  - Idempotency keys for side effects
  - Sleep and wait for events
  - Workflow SDK, Temporal or Inngest
  - LangGraph checkpointers
  - Workflow SDK: Foundations
  - LangGraph: Persistence
- Queues and Background Jobs
  Core
  Long agent runs do not fit inside an HTTP request, so they move to queues and workers that report progress back to the user.
  - Move runs off the request path
  - Job queues such as BullMQ
  - Progress over SSE or websockets
  - Concurrency limits per user
  - Dead-letter queues for failures
  - BullMQ docs
- Long-running and Scheduled Agents
  Recommended
  Agents that run on a schedule or on events, and may wait days for approval, need persistent sessions and firm budgets.
  - Cron and webhook triggers
  - Persisted, resumable sessions
  - Approvals that wait for days
  - Budgets and limits per run
  - Notify users when done
  - AI SDK: WorkflowAgent
  - Claude Agent SDK: Sessions
- Managed Agent Runtimes
  Optional
  Providers now host complete agent harnesses with sandboxes and sessions, trading some control for far less infrastructure work.
  - Hosted loop, sandbox and sessions
  - Compare with self-hosted SDK agents
  - Data residency and lock-in
  - Pricing per session and tokens
  - Claude docs: Managed Agents
  - OpenAI docs: Agents API
- Agent-to-agent Protocols
  Optional
  A2A lets agents built by different teams or vendors discover each other and exchange tasks, which complements MCP's agent-to-tool role.
  - Agent cards for discovery
  - Tasks and messages between agents
  - How A2A and MCP differ
  - Signed cards and trust
  - A2A protocol docs
Build: Parallel Research Pipeline
Build a research agent whose lead agent splits a question into subtasks, runs three to five subagents in parallel as durable steps, survives a server restart mid-run, and emails a cited report when it finishes.
8
Evals and Observability
About 3-4 weeks
Measure agent quality with repeatable evals, catch regressions in CI and trace every production run down to each tool call.
- Success Criteria and Golden Datasets
  Core
  You cannot improve an agent you cannot measure, and a curated set of real inputs with expected outcomes is where every eval starts.
  - Measurable success criteria
  - Real user inputs, not invented ones
  - Expected outputs or rubrics
  - Edge cases and known failures
  - Add every production bug
  - Claude docs: Define success and build evals
- Code-based Checks
  Core
  Deterministic checks are cheap, fast and unambiguous, so use them wherever code can verify the correct result.
  - Exact match and regex checks
  - Schema validation of outputs
  - Expected tool call sequences
  - Final state checks in a sandbox
  - Unit tests with mock models
  - AI SDK: Testing
- LLM-as-judge
  Core
  Open-ended quality needs a model grader, and a judge is only useful once you have checked that it agrees with human ratings.
  - Rubric-based grading prompts
  - Pairwise comparisons
  - Calibrate against human labels
  - Measure judge agreement
  - Watch position and length bias
  - OpenAI docs: Evals
- Evaluating Agents End to End
  Core
  Agents are non-deterministic and take many steps, so you grade task success across repeated trials rather than one transcript.
  - Task success over many trials
  - pass@k and consistency
  - Reviewing full trajectories
  - Steps and cost per task
  - Clean environment per trial
- Regression Evals in CI
  Core
  Prompt edits and model upgrades quietly break behavior, so evals should run on every change like any other test suite.
  - Run evals on each prompt change
  - Score thresholds that fail builds
  - Compare models before switching
  - Cache results to control cost
  - promptfoo, Evalite or Braintrust
  - promptfoo docs
- Tracing with OpenTelemetry
  Core
  Traces show every model call, tool call and retry inside a run, which is the only practical way to debug agents in production.
  - Spans for model and tool calls
  - GenAI semantic conventions
  - Trace context across services
  - AI SDK telemetry settings
  - Langfuse, Phoenix or LangSmith
  - Redact sensitive data in traces
  - OpenTelemetry: GenAI conventions
  - AI SDK: Telemetry
- Cost and Latency Tracking
  Recommended
  Agents can multiply token spend without warning, so track cost and latency per user, feature and model from the first release.
  - Tokens per request and user
  - Cache hit rate
  - p50 and p95 latency
  - Time to first token
  - Alerts on cost spikes
  - Claude Agent SDK: Cost tracking
Build: Eval Suite in CI
Add a 50-case eval suite to your triage or docs agent with code checks and a calibrated LLM judge, and run it in GitHub Actions on every pull request. Export traces from each run to an OpenTelemetry backend.
9
Safety, Security and Production
About 4-5 weeks
Ship agents that resist prompt injection, run with least privilege, stay within budget and keep working when a provider fails.
- Prompt Injection and the OWASP Lists
  Core
  Any text an agent reads can carry instructions, so prompt injection is the top risk for tool-using agents and has no complete fix.
  - Direct vs indirect injection
  - Untrusted content in tool results
  - Exfiltration through links and images
  - OWASP Top 10 for LLM apps
  - OWASP Top 10 for agentic apps
  - Red-team your own agent
  - OWASP: LLM Top 10 2026
  - OWASP: Top 10 for Agentic Applications
- Least-privilege Tools and Identity
  Core
  Limiting what each agent can reach contains the damage when an injection or a bad plan gets through.
  - Scoped, short-lived credentials
  - Act as the user, not admin
  - Read-only by default
  - Allowlists for domains and commands
  - Isolate agents reading untrusted input
  - Audit log of every action
  - Claude Agent SDK: Permissions
  - Claude Agent SDK: Secure deployment
- Sandboxed Code Execution
  Core
  Agents that write and run code must do it in disposable, isolated environments, never directly on your servers or a user's machine.
  - Never run model code on the host
  - MicroVMs or containers per session
  - Network egress controls
  - CPU, memory and time limits
  - Ephemeral file systems
  - Vercel docs: Sandbox
  - E2B docs
- Output Validation and Guardrails
  Core
  Model output is untrusted input to the rest of your system, so validate it and run guardrail checks before it is shown or acted on.
  - Validate every tool argument
  - Input and output guardrail checks
  - Moderation and policy filters
  - Escape output for HTML and SQL
  - Block secrets and PII in replies
  - OpenAI Agents SDK: Guardrails
  - Claude docs: Mitigate jailbreaks
- PII and Data Handling
  Recommended
  Prompts, traces and memories collect personal data quickly, so data minimization and retention settings need a decision before launch.
  - Send models only needed data
  - Redact PII before logging
  - Provider data retention settings
  - Regional data residency
  - Honor user deletion requests
  - Claude docs: API and data retention
  - OpenAI docs: Data controls
- Rate Limits, Fallbacks and Routing
  Core
  Providers throttle and have outages, so production agents need retries, fallbacks and routing of easy work to cheaper models.
  - Handle 429 and overloaded errors
  - Retries with jitter
  - Fallback to another provider
  - Route easy tasks to small models
  - Timeouts and circuit breakers
  - AI gateways for failover
  - Claude docs: Rate limits
  - Vercel docs: AI Gateway
- Deployment and Cost Control
  Core
  Agents differ from normal web requests in runtime, streaming and spend, so hosting and quotas need their own plan.
  - Serverless vs long-running containers
  - Streaming through proxies and CDNs
  - Per-user quotas and billing
  - Batch APIs for offline work
  - Feature flags for model changes
  - Incident runbooks
  - Claude Agent SDK: Hosting
  - Claude docs: Batch processing
Build: Production Launch
Ship one of your agents publicly with per-user auth, sandboxed code execution, a prompt injection test suite, provider fallback, per-user quotas and a cost dashboard. Then write a one-page incident runbook.

Last reviewed Sep 25, 2026.

Foundations for Agent Work

LLM Fundamentals

Prompt and Context Engineering

Tool Use and the Agent Loop

Knowledge, Retrieval and Memory

MCP and Integrations

Multi-agent Systems and Durable Workflows

Evals and Observability

Safety, Security and Production

Foundations for Agent Work

TypeScript Essentials

Async Code and Streams

HTTP APIs and Server-Sent Events

JSON and JSON Schema

Git, Secrets and Project Setup

Python as a Second Language

Math Intuition

LLM Fundamentals

How LLMs Generate Text

The Messages API

Generation Settings

Streaming Responses

Choosing Models

Multimodal Inputs

Provider-agnostic SDKs

Prompt and Context Engineering

Writing Clear Instructions

Structured Outputs

Context Engineering

Prompt Caching

Token Counting and Budgets

Grounding and Hallucination Control

Tool Use and the Agent Loop

Tool Calling

Designing Good Tools

The Agent Loop

Workflows vs Agents

Error Handling and Recovery

Human-in-the-loop Approval

Agent SDKs and Frameworks

Knowledge, Retrieval and Memory

Embeddings

Ingestion and Chunking

Vector Search in Your Database

Hybrid Search and Reranking

Agentic Retrieval and Citations

Evaluating Retrieval

Short and Long-Term Memory

MCP and Integrations

MCP Architecture

Building an MCP Server

MCP Auth and Security

Connecting MCP to Agents

Agent Skills

MCP Extensions and User Input

Browser and Computer Use

Multi-agent Systems and Durable Workflows

Orchestrators and Subagents

Handoffs and Routing

Durable Execution

Queues and Background Jobs

Long-running and Scheduled Agents

Managed Agent Runtimes

Agent-to-agent Protocols

Evals and Observability

Success Criteria and Golden Datasets

Code-based Checks

LLM-as-judge

Evaluating Agents End to End

Regression Evals in CI

Tracing with OpenTelemetry

Cost and Latency Tracking

Safety, Security and Production

Prompt Injection and the OWASP Lists

Least-privilege Tools and Identity

Sandboxed Code Execution

Output Validation and Guardrails

PII and Data Handling

Rate Limits, Fallbacks and Routing

Deployment and Cost Control