Building a Low-Code Multi-Agent Router with LangChain and LangGraph
March 20, 2026
Building a Low-Code Multi-Agent Router with LangChain and LangGraph
A walkthrough of a production implementation with a runnable example
When building production AI systems, you quickly discover that one-size-fits-all agents don't cut it. A coding assistant needs different tools and prompts than a documentation bot. A data analyst agent requires different capabilities than a customer support agent.
The solution? Multi-agent architectures where a router intelligently directs queries to specialized sub-agents.
In this article, I'll walk through a real production codebase that implements this pattern, then give you a complete runnable script you can adapt for your own projects.
The Architecture at a Glance
The key insight: the router itself is just another LLM call that outputs structured data. No complex rule engines. No pattern matching. Just let the LLM decide.
Walking Through the Project
Let's trace through each layer of the implementation, starting from the bottom up.
Layer 1 Specialized Sub-Agents
Each sub-agent is focused on a specific domain. Here's an example agent:
1from langchain.agents import create_agent
2from langchain_aws import ChatBedrock
3
4def create_my_favorite_agent() -> CompiledStateGraph:
5 """Create my_favorite_agent."""
6 return create_agent(
7 name="my_favorite_agent",
8 model="your-favorite-model",
9 system_prompt="Your specialized prompt goes here.",
10 tools=[
11 ...,
12 ],
13 )
14
15def create_my_other_favorite_agent() -> CompiledStateGraph:
16 """Create my_other_favorite_agent."""
17 return create_agent(
18 name="my_other_favorite_agent",
19 model="your-favorite-model",
20 system_prompt="Your other specialized prompt goes here.",
21 tools=[
22 ...,
23 ],
24 )Notice each agent has:
- Its own focused prompt
- Its own tool set
Layer 2: The Router Agent
Here's where the magic happens. The router uses structured output to make routing decisions:
1from pydantic import BaseModel, Field
2from langgraph.graph import END, START, StateGraph
3
4class Route(TypedDict):
5 """Defines a route (agent node) in the graph."""
6 name: str
7 description: str
8 runnable: Runnable[Any, Any] # any agent or graph :)
9
10
11class RouteDecision(BaseModel):
12 """Router's structured output for deciding which agent to invoke."""
13 node_name: str = Field(description="Name of the node to route to")The router prompt is dynamically generated from route descriptions:
1def create_routing_subagent(
2 routes: list[Route],
3 model: BaseChatModel
4) -> CompiledStateGraph:
5 """Build router with auto-generated prompt from routes."""
6
7 system_prompt = (
8 "You are a helpful assistant responsible for routing the query "
9 "to the correct graph node. Given the user query, select the best "
10 "available route from the routes below:\n\n"
11 )
12
13 for route in routes:
14 system_prompt += f"\n\nRoute Name: {route['name']}"
15 system_prompt += f"\nDescription: {route['description']}"
16
17 return create_agent(
18 name="router_agent",
19 model=model,
20 response_format=RouteDecision, # Structured output!
21 system_prompt=system_prompt,
22 )Then we wire up the graph with conditional edges:
1def create_routing_agent(
2 model: BaseChatModel,
3 routes: list[Route],
4) -> CompiledStateGraph:
5 """Build the complete router graph."""
6
7 graph = StateGraph(State)
8 router_agent = create_routing_subagent(routes, model)
9
10 # Router node: invoke and store decision
11 async def route_node(state: State) -> dict[str, Any]:
12 result = await router_agent.ainvoke(state)
13 route_decision = RouteDecision.model_validate(
14 result["structured_response"]
15 )
16 return {"route_decision": route_decision}
17
18 graph.add_node(router_agent.name, route_node)
19
20 # Add all route destinations
21 node_names = []
22 for route in routes:
23 graph.add_node(route["name"], route["runnable"])
24 graph.add_edge(route["name"], END)
25 node_names.append(route["name"])
26
27 # Routing function reads decision from state
28 def _route(state: State) -> str:
29 return state["route_decision"].node_name
30
31 graph.add_edge(START, router_agent.name)
32 graph.add_conditional_edges(router_agent.name, _route, node_names)
33
34 return graph.compile(name="routing_agent")Layer 5: The Complete Research Assistant
Finally, we compose everything into the top-level graph:
1def create_agent(cfg: Config) -> CompiledStateGraph:
2 """Create complete research assistant with routing."""
3
4 docs_provider = MarkdownDocsProvider(cfg.docs_path)
5
6 # Create specialized sub-agents
7 my_favorite_agent = create_my_favorite_agent()
8 my_other_favorite_agent = create_my_other_favorite_agent()
9
10 # Create router with route definitions
11 return create_routing_agent(
12 model="your favorite FAST model", # use a fast thinking model here :)
13 routes=[
14 {
15 "name": my_favorite_agent.name,
16 "description": (
17 "Use for queries that require code generation:\n"
18 "- Writing Python code for cohort creation\n"
19 "- SQL queries for OMOP data\n"
20 "- Code modifications, debugging help\n"
21 '- "Write code to..." requests\n'
22 "- Any question where code would help"
23 ),
24 "runnable": my_favorite_agent,
25 },
26 {
27 "name": my_other_favorite_agent.name,
28 "description": (
29 "Use for documentation and conceptual questions:\n"
30 "- Setup and configuration help\n"
31 '- Conceptual explanations ("What is...")\n'
32 "- Data model questions (without code)"
33 ),
34 "runnable": my_other_favorite_agent,
35 },
36 ],
37 )That's it—a complete multi-agent system in ~200 lines of actual code.
Context Engineering: The Hidden Superpower
There's a term gaining traction in the AI engineering world: context engineering. It's the discipline of carefully crafting what information goes into the LLM's context window—and crucially, what stays out.
The multi-agent router pattern is fundamentally a context engineering architecture.
The Problem with Monolithic Prompts
The naive approach is one agent with one massive prompt covering everything. Problems:
- Context bloat: Thousands of irrelevant tokens on every request
- Instruction interference: Guidelines for coding conflict with Q&A guidelines
- Degraded performance: LLMs perform worse as context grows
- Maintenance nightmare: One change risks breaking unrelated functionality
The Router as Context Curator
With the router pattern:
Each agent sees only the context relevant to its specialty.
Route Descriptions Are Context Engineering
Those route descriptions aren't just documentation—they're the context that enables accurate routing:
1# Vague description → misrouting
2"description": "Handles code stuff"
3
4# Precise description → accurate routing
5"description": (
6 "Use for queries requiring executable code:\n"
7 "- Python scripts and functions\n"
8 "- SQL queries for data access\n"
9 '- Requests starting with "Write code to..."'
10)The description is context engineering for the router itself.
Conclusion
Building multi-agent systems doesn't have to be complicated. The pattern we've walked through:
- Specialized sub-agents with focused prompts and tools
- A router agent using structured output for classification
- Dynamic prompt generation from route descriptions
- Tiered model selection (fast for routing, smart for work)
- State-based bypass when context is already known
The result: a system where adding a new capability is a matter of defining a new agent and adding a route description. The router learns to use it automatically.
Start small—two agents and a router. You can always add more routes later.
The code in this article is adapted from a production system. The runnable example is simplified but demonstrates all the core patterns.