AI Agent Tools

Tauhid Zaman | Yale SOM

The "Static" Chatbot Problem

Standard chatbots are like brilliant scholars locked in a room.

  • They can talk about anything...
  • But they can't do anything.
  • They have no "hands" to act upon your commands.

The Solution: Tools

We give the chatbot access to a set of tools

🗣️

Natural Language

🛠️

Tool Calling

⚙️

Automated Action

The 4-Step Handshake

How a chat becomes action

  1. Tool declaration: Define the tool name and parameters.
  2. Reasoning: Chatbot decides to use the tool.
  3. The Call: Chatbot returns parameters as a JSON object (not text).
  4. Execution: Your app runs a function with the parameters.

(Press Down ↓ to see the code contract)

Step 1: Tool Declaration


{
  "name": "generateMovieScript",
  "description": "Updates the scene editor with a new script",
  "parameters": {
    "type": "object",
    "properties": {
      "scenes": {
        "type": "array",
        "items": { 
          "type": "object", 
          "properties": { "description": "string", "narration": "string" } 
        }
      }
    },
    "required": ["scenes"]
  }
}
                    

Step 2: Chatbot Response

The "Invisible Handshake"


{
  "functionCall": {
    "name": "generateMovieScript",
    "args": {
      "scenes": [
        { "description": "Neon city...", "narration": "Welcome..." }
      ]
    }
  }
}
                    

Step 3: "Catch & Execute" Function

How the app handles the chatbot's request.


async function handleChat(userInput) {
  const result = await chat.sendMessage(userInput);
  
  // 1. Check if Gemini requested a Tool
  const call = result.response.functionCalls()?.[0];

  if (call?.name === "generateMovieScript") {
    
    // 2. EXECUTION: Run the local React function
    handleAssistantScript(call.args.scenes); 

    return "I've updated the editor for you!";
  }
  
  return result.response.text();
}
                    
Action: The code then executes the handleAssistantScript function with the parameters.

User Experience: The Assistant

Hi! How can I help?
Write a 5 scene script for ...
... ⚙️
The script is ready!
  • The chatbot identified a creative intent.
  • It calculated the parameters.
  • Your app executed the handleAssistantScript function with the parameters.

Traditional User Interface (UI) vs AI Tools

Feature Traditional UI AI Tools
Input Forms & Buttons Conversation
Accuracy Manual Data Entry AI Assisted Data Generation
Flexibility Rigid Workflow Dynamic Workflow

AI Tool Examples

  • Content Creation Assistant: Agent creates and posts social media content for you.
  • Web Search: Agent searches the web to answer questions or provide information.
  • Calendar Management: Agent schedules your meetings on your calendar.
  • Data Scientist: Agent uses a run_python() tool to generate charts, run regressions, and analyze the results.
  • Portfolio Manager: Agent calls a trade_crypto() tool to execute buy/sell orders based on a strategy.

Key Takeaways

  • Chatbots > Agents : Tools turn chatbots into active assistants.
  • Reliability: AI tools make user experience easier and more reliable
  • Flexibility: Chatbot interface makes the user experience dynamic and personalized
  • Power: AI tools let you become a super human if used creatively