Building AI Agents with LangChain & Tool Calling

In the previous blog, we built a RAG pipeline that retrieves and answers questions from documents.

But what if your AI needs to:

• Fetch live data from APIs
• Trigger workflows
• Query databases
• Perform actions instead of just answering

This is where AI Agents + Tool Calling come in.

From RAG to Agents

RAG is: “Find relevant information → Generate answer”

Agents are: “Understand goal → Decide action → Use tools → Iterate → Produce result”

The key shift:

LLM is no longer just answering — it is deciding

What Exactly is an AI Agent?

Before jumping into implementation, it’s important to understand what we really mean by an AI Agent.

An AI agent is not just a chatbot.

It is a system where:

• A language model acts as the reasoning engine
• External tools provide capabilities
• And a loop enables decision → action → observation

In simple terms:

An agent is an LLM that doesn’t just respond — it decides and acts

Unlike traditional LLM applications that generate a single response, agents operate in multi-step workflows where they can:

• Break down problems
• Choose tools dynamically
• Execute actions
• Iterate until the goal is achieved

⚙️ What is Tool Calling?

Tool calling allows an LLM to:

1. See available tools
2. Decide which tool to use
3. Generate arguments
4. Call the tool
5. Use the result to continue reasoning

In LangChain, this is abstracted cleanly.

Why Tool Calling is the Core of Agents

Tool calling is what transforms an LLM from:
— a text generator
into
— a system that can interact with the real world

Without tools, an LLM is limited to:

• Its training data
• Static reasoning

With tools, it can:

• Query live APIs
• Access databases
• Trigger workflows
• Perform computations

Technically, tool calling works by:

1. Providing the LLM with a list of tools (functions)
2. Describing what each tool does (via docstrings)
3. Letting the model decide when and how to call them

Modern LLMs can even return:

• Structured tool calls
• Arguments for execution
• Instead of plain text responses

This is why tool descriptions are extremely critical — they act as the interface between reasoning and execution.

Real-World Example

Let’s build a simple agent:

Use Case: DevOps Assistant

User asks: “Check if my service is down and restart it if needed”

This requires:

• Checking service status (API/tool)
• Restarting service (another tool)

Not something an LLM can hallucinate — it must act.

🏗️ Architecture

User Query →
Agent →
   decides →
      Tool A (check status)
      Tool B (restart service)
→ Final response

🧰 Step 1: Define Tools

In LangChain, tools are just Python functions.

from langchain.tools import tool

@tool
def check_service(service_name: str) -> str:
    """Check if a service is running"""
    # mock logic
    return "down"

@tool
def restart_service(service_name: str) -> str:
    """Restart a service"""
    return f"{service_name} restarted successfully"

🧠 Step 2: Initialize LLM

from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(model="gpt-4")

🔗 Step 3: Create Agent

from langchain.agents import initialize_agent

tools = [check_service, restart_service]

agent = initialize_agent(
    tools,
    llm,
    agent="zero-shot-react-description",
    verbose=True
)

💬 Step 4: Run the Agent

response = agent.run(
    "Check if nginx is running and restart if it is down"
)

print(response)

The Agent Execution Loop (Important Concept)

Most agent systems follow a loop like this:

User Input →
Reason →
Select Tool →
Execute Tool →
Observe Result →
Repeat (if needed) →
Final Answer

This is often referred to as a reasoning + action loop.

Why this matters:

• The agent is not solving everything in one step
• It is iteratively improving its understanding
• Each tool call becomes new context

This loop is what enables:

• Multi-step workflows
• Conditional execution
• Real-world automation

🔍 What Happens Internally

1. LLM reads tool descriptions
2. Breaks down the problem
3. Calls check_service("nginx")
4. Gets result → "down"
5. Decides next step
6. Calls restart_service("nginx")
7. Returns final answer

How Does the LLM Decide Which Tool to Use?

This is one of the most misunderstood parts.

The LLM does NOT:

• “know” tools like a program
• follow strict rules (unless enforced)

Instead, it uses:

• Natural language understanding
• Tool descriptions
• Prompt context

To make a probabilistic decision.

For example:

If the user says: “restart nginx if it’s down”

The model internally reasons like:

• I need to check status → tool available? yes
• If down → restart → tool available? yes

This reasoning is guided by:

• Prompt structure
• Tool naming
• Examples (if provided)

Which means:

👉 Better descriptions = better decisions

Making This Deterministic (Important for Engineers)

This is where most people get it wrong.

Naive Thinking:

“LLM will figure it out”

Correct Approach:

You guide the decision boundaries

Control Levers:

1. Tool Descriptions

Bad: “restart service”
Good: “Restart a service only if it is confirmed to be down”

2. System Prompt

agent = initialize_agent(
    tools,
    llm,
    agent="zero-shot-react-description",
    agent_kwargs={
        "system_message": "Always check service status before restarting"
    }
)

3. Limit Tool Access

Only expose what’s necessary. Too many tools can lead to llm picking the incorrect tool for the job.

4. Add Guardrails

• Validation layers
• Rate limits
• Confirmation steps

Key Takeaways

• Tool calling turns LLMs into decision engines
• Agents are powerful but less deterministic than RAG
• Control comes from:
  - prompts
  - tool definitions
  - constraints