HomeSpecialized Use CasesBuilding Chatbot Frameworks

intermediate15 min read· Module 13, Lesson 2

🗨️Building Chatbot Frameworks

Multi-turn state machines, intent routing, and human handoff

Building Chatbot Frameworks

A simple chatbot answers one question and forgets you exist. A production chatbot framework manages conversations as living processes — tracking where the user is in a workflow, detecting what they want, filling in missing information, and knowing when to give up and call a human. This lesson teaches you how to build that kind of system from the ground up.

Beyond Simple Chat: Why Frameworks Matter

Most tutorials show you how to send a message to an LLM and get a response. That is not a chatbot. A real chatbot needs to handle:

Multi-turn conversations where context from message 1 affects message 15
Branching logic where the user's answer changes the entire flow
Slot filling where the bot needs to collect 5 pieces of information before it can act
Graceful failure when the user says something completely unexpected
Human handoff when the bot hits its limits
Session persistence so the user can leave and come back tomorrow

Without a framework, you end up with spaghetti code — hundreds of if/else blocks that nobody can maintain. A framework gives you structure, testability, and the ability to scale.

Conversation State Machines

The foundation of any serious chatbot is a state machine. Each state represents a point in the conversation, and transitions define how the bot moves between states based on user input.

Core Concepts

A conversation state machine has four components:

States — Named stages in the conversation (e.g., greeting, collecting_email, confirming_order)
Transitions — Rules that move the conversation from one state to another
Guards — Conditions that must be true for a transition to fire
Actions — Side effects that happen during a transition (e.g., saving data, calling an API)

Designing a State Machine

TypeScript
// Define the states for a customer support bot
type BotState =
  | "idle"
  | "greeting"
  | "identify_intent"
  | "collect_order_id"
  | "lookup_order"
  | "provide_status"
  | "collect_issue"
  | "escalate_to_human"
  | "collect_feedback"
  | "farewell";

// Define the context that persists across states
interface ConversationContext {
  userId: string | null;
  orderId: string | null;
  intent: string | null;
  issueDescription: string | null;
  turnCount: number;
  lastActivity: number;
  slotsFilled: Record<string, string>;
  history: Array<{ role: string; content: string }>;
}

// Define transitions
interface Transition {
  from: BotState;
  to: BotState;
  guard?: (ctx: ConversationContext, input: string) => boolean;
  action?: (ctx: ConversationContext, input: string) => Promise<void>;
}

Implementing the State Machine

TypeScript
class ConversationStateMachine {
  private state: BotState = "idle";
  private context: ConversationContext;
  private transitions: Transition[] = [];

  constructor(initialContext: Partial<ConversationContext> = {}) {
    this.context = {
      userId: null,
      orderId: null,
      intent: null,
      issueDescription: null,
      turnCount: 0,
      lastActivity: Date.now(),
      slotsFilled: {},
      history: [],
      ...initialContext,
    };
  }

  addTransition(transition: Transition): void {
    this.transitions.push(transition);
  }

  async processInput(input: string): Promise<string> {
    this.context.turnCount++;
    this.context.lastActivity = Date.now();
    this.context.history.push({ role: "user", content: input });

    // Find the first matching transition from current state
    const validTransitions = this.transitions.filter(
      (t) => t.from === this.state
    );

    for (const transition of validTransitions) {
      const guardPassed = transition.guard
        ? transition.guard(this.context, input)
        : true;

      if (guardPassed) {
        // Execute the action if defined
        if (transition.action) {
          await transition.action(this.context, input);
        }

        // Transition to new state
        this.state = transition.to;

        // Generate response based on new state
        const response = await this.generateResponse();
        this.context.history.push({ role: "assistant", content: response });
        return response;
      }
    }

    // No valid transition found — fallback
    return this.handleFallback(input);
  }

  private async generateResponse(): Promise<string> {
    // Each state has a corresponding response generator
    const responseMap: Record<BotState, () => string> = {
      idle: () => "Hello! How can I help you today?",
      greeting: () =>
        "Welcome back! What can I assist you with?",
      identify_intent: () =>
        "I can help with order tracking, returns, or general questions. What do you need?",
      collect_order_id: () =>
        "Sure, I can look that up. What is your order ID?",
      lookup_order: () =>
        `Looking up order ${this.context.orderId}...`,
      provide_status: () =>
        `Your order ${this.context.orderId} is being processed.`,
      collect_issue: () =>
        "Can you describe the issue you are experiencing?",
      escalate_to_human: () =>
        "Let me connect you with a human agent who can help further.",
      collect_feedback: () =>
        "Was this helpful? Please rate your experience.",
      farewell: () =>
        "Thank you for contacting us. Have a great day!",
    };
    return responseMap[this.state]();
  }

  private handleFallback(input: string): string {
    return "I did not quite understand that. Could you rephrase?";
  }

  getState(): BotState {
    return this.state;
  }

  getContext(): ConversationContext {
    return { ...this.context };
  }
}

Intent Detection and Routing

Once you have a state machine, you need to figure out what the user wants. This is intent detection — classifying user messages into actionable categories.

Pattern-Based Intent Detection

For simple bots, regex patterns work fine:

TypeScript
interface IntentPattern {
  intent: string;
  patterns: RegExp[];
  confidence: number;
}

const intentPatterns: IntentPattern[] = [
  {
    intent: "track_order",
    patterns: [
      /where.*(my|the).*order/i,
      /track.*order/i,
      /order.*status/i,
      /shipping.*update/i,
    ],
    confidence: 0.85,
  },
  {
    intent: "request_refund",
    patterns: [
      /refund/i,
      /money.*back/i,
      /return.*order/i,
      /cancel.*order/i,
    ],
    confidence: 0.80,
  },
  {
    intent: "speak_to_human",
    patterns: [
      /speak.*human/i,
      /talk.*agent/i,
      /real.*person/i,
      /transfer.*me/i,
    ],
    confidence: 0.95,
  },
];

function detectIntent(
  message: string
): { intent: string; confidence: number } | null {
  for (const pattern of intentPatterns) {
    for (const regex of pattern.patterns) {
      if (regex.test(message)) {
        return { intent: pattern.intent, confidence: pattern.confidence };
      }
    }
  }
  return null;
}

LLM-Based Intent Detection

For production systems, use Claude to classify intents with structured output:

TypeScript
async function detectIntentWithLLM(
  message: string,
  conversationHistory: Array<{ role: string; content: string }>
): Promise<{ intent: string; confidence: number; entities: Record<string, string> }> {
  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-20250514",
    max_tokens: 256,
    system: `You are an intent classifier for a customer support bot.
Classify the user message into one of these intents:
- track_order: user wants to check order status
- request_refund: user wants money back or to return
- product_question: user asks about a product
- speak_to_human: user wants a real agent
- greeting: user is saying hello
- farewell: user is saying goodbye
- unknown: cannot determine intent

Also extract any entities (order_id, product_name, email).
Respond ONLY in JSON: {"intent": "...", "confidence": 0.0-1.0, "entities": {}}`,
    messages: [
      ...conversationHistory.map((m) => ({
        role: m.role as "user" | "assistant",
        content: m.content,
      })),
      { role: "user", content: message },
    ],
  });

  const text = response.content[0].type === "text"
    ? response.content[0].text
    : "";
  return JSON.parse(text);
}

Intent Router

Route detected intents to the correct handler:

TypeScript
class IntentRouter {
  private handlers: Map<string, (ctx: ConversationContext, entities: Record<string, string>) => Promise<string>> = new Map();

  register(
    intent: string,
    handler: (ctx: ConversationContext, entities: Record<string, string>) => Promise<string>
  ): void {
    this.handlers.set(intent, handler);
  }

  async route(
    intent: string,
    ctx: ConversationContext,
    entities: Record<string, string>
  ): Promise<string> {
    const handler = this.handlers.get(intent);
    if (!handler) {
      return this.defaultHandler(ctx);
    }
    return handler(ctx, entities);
  }

  private defaultHandler(ctx: ConversationContext): string {
    return "I am not sure how to help with that. Could you tell me more?";
  }
}

Context Management Across Turns

A chatbot that forgets what you said two messages ago is useless. Context management is how you maintain a coherent conversation across many turns.

The Context Store

TypeScript
class ConversationContextStore {
  private sessions: Map<string, ConversationContext> = new Map();
  private readonly maxAge = 30 * 60 * 1000; // 30-minute session timeout

  getOrCreate(sessionId: string): ConversationContext {
    const existing = this.sessions.get(sessionId);

    if (existing && Date.now() - existing.lastActivity < this.maxAge) {
      return existing;
    }

    // Create fresh context
    const newContext: ConversationContext = {
      userId: null,
      orderId: null,
      intent: null,
      issueDescription: null,
      turnCount: 0,
      lastActivity: Date.now(),
      slotsFilled: {},
      history: [],
    };

    this.sessions.set(sessionId, newContext);
    return newContext;
  }

  update(sessionId: string, updates: Partial<ConversationContext>): void {
    const ctx = this.sessions.get(sessionId);
    if (ctx) {
      Object.assign(ctx, updates, { lastActivity: Date.now() });
    }
  }

  cleanup(): void {
    const now = Date.now();
    for (const [id, ctx] of this.sessions) {
      if (now - ctx.lastActivity > this.maxAge) {
        this.sessions.delete(id);
      }
    }
  }
}

Sliding Window for Token Management

You cannot send the entire conversation history to the LLM forever. Use a sliding window:

TypeScript
function buildContextWindow(
  history: Array<{ role: string; content: string }>,
  maxTurns: number = 20,
  systemSummary?: string
): Array<{ role: string; content: string }> {
  if (history.length <= maxTurns) {
    return history;
  }

  // Keep the most recent turns
  const recentHistory = history.slice(-maxTurns);

  // Optionally prepend a summary of older conversation
  if (systemSummary) {
    return [
      { role: "assistant", content: `[Previous conversation summary: ${systemSummary}]` },
      ...recentHistory,
    ];
  }

  return recentHistory;
}

Slot Filling

Slot filling is the process of collecting all required pieces of information before the bot can take an action. Think of it like a form that gets filled in through conversation.

TypeScript
interface Slot {
  name: string;
  prompt: string;
  required: boolean;
  validate: (value: string) => boolean;
  extract: (message: string) => string | null;
}

class SlotFiller {
  private slots: Slot[];
  private filled: Record<string, string> = {};

  constructor(slots: Slot[]) {
    this.slots = slots;
  }

  processMessage(message: string): {
    complete: boolean;
    nextPrompt: string | null;
    filledSlots: Record<string, string>;
  } {
    // Try to extract slot values from the message
    for (const slot of this.slots) {
      if (!this.filled[slot.name]) {
        const extracted = slot.extract(message);
        if (extracted && slot.validate(extracted)) {
          this.filled[slot.name] = extracted;
        }
      }
    }

    // Check if all required slots are filled
    const missingRequired = this.slots.filter(
      (s) => s.required && !this.filled[s.name]
    );

    if (missingRequired.length === 0) {
      return {
        complete: true,
        nextPrompt: null,
        filledSlots: { ...this.filled },
      };
    }

    // Ask for the next missing slot
    return {
      complete: false,
      nextPrompt: missingRequired[0].prompt,
      filledSlots: { ...this.filled },
    };
  }

  reset(): void {
    this.filled = {};
  }
}

// Example: Refund request slot filler
const refundSlots: Slot[] = [
  {
    name: "order_id",
    prompt: "What is your order ID? It starts with ORD-.",
    required: true,
    validate: (v) => /^ORD-d{6,}$/.test(v),
    extract: (msg) => {
      const match = msg.match(/ORD-d{6,}/);
      return match ? match[0] : null;
    },
  },
  {
    name: "reason",
    prompt: "What is the reason for the refund?",
    required: true,
    validate: (v) => v.length > 10,
    extract: (msg) => (msg.length > 10 ? msg : null),
  },
  {
    name: "email",
    prompt: "What email address should we send the confirmation to?",
    required: true,
    validate: (v) => /^[^@]+@[^@]+.[^@]+$/.test(v),
    extract: (msg) => {
      const match = msg.match(/[^s@]+@[^s@]+.[^s@]+/);
      return match ? match[0] : null;
    },
  },
];

Escalation to Humans

No chatbot can handle everything. A good framework knows when to escalate and does it gracefully.

Escalation Triggers

TypeScript
interface EscalationRule {
  name: string;
  check: (ctx: ConversationContext, message: string) => boolean;
  priority: "low" | "medium" | "high" | "critical";
  reason: string;
}

const escalationRules: EscalationRule[] = [
  {
    name: "explicit_request",
    check: (ctx, msg) => /speak.*human|real.*person|agent/i.test(msg),
    priority: "high",
    reason: "Customer explicitly requested a human agent",
  },
  {
    name: "frustration_detected",
    check: (ctx, msg) => {
      const frustrationWords = ["useless", "terrible", "worst", "angry", "frustrated", "ridiculous"];
      return frustrationWords.some((w) => msg.toLowerCase().includes(w));
    },
    priority: "high",
    reason: "Customer frustration detected in language",
  },
  {
    name: "too_many_turns",
    check: (ctx) => ctx.turnCount > 15,
    priority: "medium",
    reason: "Conversation exceeded 15 turns without resolution",
  },
  {
    name: "repeated_fallback",
    check: (ctx) => {
      const recentMessages = ctx.history.slice(-6);
      const fallbackCount = recentMessages.filter(
        (m) => m.role === "assistant" && m.content.includes("did not quite understand")
      ).length;
      return fallbackCount >= 3;
    },
    priority: "high",
    reason: "Bot failed to understand user 3 times in recent messages",
  },
  {
    name: "sensitive_topic",
    check: (ctx, msg) => {
      const sensitivePatterns = [/legal/i, /lawsuit/i, /lawyer/i, /complaint.*formal/i];
      return sensitivePatterns.some((p) => p.test(msg));
    },
    priority: "critical",
    reason: "Sensitive or legal topic detected",
  },
];

function checkEscalation(
  ctx: ConversationContext,
  message: string
): EscalationRule | null {
  // Sort by priority — critical first
  const priorityOrder = { critical: 0, high: 1, medium: 2, low: 3 };
  const sorted = [...escalationRules].sort(
    (a, b) => priorityOrder[a.priority] - priorityOrder[b.priority]
  );

  for (const rule of sorted) {
    if (rule.check(ctx, message)) {
      return rule;
    }
  }
  return null;
}

Handoff Protocol

TypeScript
interface HandoffPayload {
  sessionId: string;
  context: ConversationContext;
  escalationReason: string;
  priority: string;
  conversationSummary: string;
  timestamp: number;
}

async function executeHandoff(
  sessionId: string,
  ctx: ConversationContext,
  rule: EscalationRule
): Promise<HandoffPayload> {
  // Generate a summary of the conversation for the human agent
  const summary = await generateConversationSummary(ctx.history);

  const payload: HandoffPayload = {
    sessionId,
    context: ctx,
    escalationReason: rule.reason,
    priority: rule.priority,
    conversationSummary: summary,
    timestamp: Date.now(),
  };

  // Send to the human agent queue
  await sendToAgentQueue(payload);

  return payload;
}

async function generateConversationSummary(
  history: Array<{ role: string; content: string }>
): Promise<string> {
  const response = await anthropic.messages.create({
    model: "claude-sonnet-4-20250514",
    max_tokens: 300,
    system: "Summarize this support conversation in 2-3 sentences for a human agent taking over.",
    messages: [
      {
        role: "user",
        content: history.map((m) => `${m.role}: ${m.content}`).join("
"),
      },
    ],
  });
  return response.content[0].type === "text" ? response.content[0].text : "";
}

async function sendToAgentQueue(payload: HandoffPayload): Promise<void> {
  // Implementation depends on your infrastructure
  // e.g., push to Redis queue, send webhook, create ticket in Zendesk
  console.log(`Handoff queued: ${payload.sessionId} [${payload.priority}]`);
}

Fallback Handling

When the bot does not understand, it should not just say "I do not understand" and leave the user stuck. Good fallback handling is an art.

Tiered Fallback Strategy

TypeScript
class FallbackHandler {
  private consecutiveFailures: number = 0;
  private readonly maxRetries: number = 3;

  async handle(
    ctx: ConversationContext,
    input: string
  ): Promise<{ response: string; shouldEscalate: boolean }> {
    this.consecutiveFailures++;

    // Tier 1: Rephrase request
    if (this.consecutiveFailures === 1) {
      return {
        response: "I did not catch that. Could you rephrase your question?",
        shouldEscalate: false,
      };
    }

    // Tier 2: Offer specific options
    if (this.consecutiveFailures === 2) {
      return {
        response:
          "I am having trouble understanding. Can you pick one of these options?
" +
          "1. Track an order
" +
          "2. Request a refund
" +
          "3. Ask a product question
" +
          "4. Speak with a human agent",
        shouldEscalate: false,
      };
    }

    // Tier 3: Escalate to human
    if (this.consecutiveFailures >= this.maxRetries) {
      return {
        response:
          "It looks like I am unable to help with this. " +
          "Let me connect you with a human agent who can assist you better.",
        shouldEscalate: true,
      };
    }

    return { response: "Please try again.", shouldEscalate: false };
  }

  reset(): void {
    this.consecutiveFailures = 0;
  }
}

Analytics and Monitoring

You cannot improve what you do not measure. Every production chatbot needs analytics.

TypeScript
interface ConversationMetrics {
  sessionId: string;
  startedAt: number;
  endedAt: number;
  totalTurns: number;
  intentsDetected: string[];
  escalated: boolean;
  escalationReason: string | null;
  resolutionStatus: "resolved" | "escalated" | "abandoned";
  feedbackScore: number | null;
  avgResponseTimeMs: number;
}

class ChatbotAnalytics {
  private metrics: ConversationMetrics[] = [];

  record(metric: ConversationMetrics): void {
    this.metrics.push(metric);
  }

  getResolutionRate(): number {
    const resolved = this.metrics.filter(
      (m) => m.resolutionStatus === "resolved"
    ).length;
    return resolved / this.metrics.length;
  }

  getEscalationRate(): number {
    const escalated = this.metrics.filter((m) => m.escalated).length;
    return escalated / this.metrics.length;
  }

  getAverageTurns(): number {
    const total = this.metrics.reduce((sum, m) => sum + m.totalTurns, 0);
    return total / this.metrics.length;
  }

  getTopIntents(limit: number = 5): Array<{ intent: string; count: number }> {
    const counts: Record<string, number> = {};
    for (const m of this.metrics) {
      for (const intent of m.intentsDetected) {
        counts[intent] = (counts[intent] || 0) + 1;
      }
    }
    return Object.entries(counts)
      .map(([intent, count]) => ({ intent, count }))
      .sort((a, b) => b.count - a.count)
      .slice(0, limit);
  }

  getAbandonmentRate(): number {
    const abandoned = this.metrics.filter(
      (m) => m.resolutionStatus === "abandoned"
    ).length;
    return abandoned / this.metrics.length;
  }
}

Multi-Channel Support

Your bot might need to work on web chat, WhatsApp, Slack, and SMS simultaneously. Abstract the channel away.

TypeScript
interface ChannelAdapter {
  channelName: string;
  sendMessage(sessionId: string, message: string): Promise<void>;
  formatMessage(message: string): string;
  parseIncoming(rawPayload: unknown): { sessionId: string; text: string };
}

class WebChatAdapter implements ChannelAdapter {
  channelName = "web";

  async sendMessage(sessionId: string, message: string): Promise<void> {
    // Push via WebSocket
  }

  formatMessage(message: string): string {
    // Supports full markdown
    return message;
  }

  parseIncoming(rawPayload: unknown): { sessionId: string; text: string } {
    const payload = rawPayload as { sessionId: string; text: string };
    return { sessionId: payload.sessionId, text: payload.text };
  }
}

class WhatsAppAdapter implements ChannelAdapter {
  channelName = "whatsapp";

  async sendMessage(sessionId: string, message: string): Promise<void> {
    // Call WhatsApp Business API
  }

  formatMessage(message: string): string {
    // Strip markdown, convert to plain text with WhatsApp formatting
    return message.replace(/**(.*?)**/g, "*$1*");
  }

  parseIncoming(rawPayload: unknown): { sessionId: string; text: string } {
    const payload = rawPayload as { from: string; body: string };
    return { sessionId: payload.from, text: payload.body };
  }
}

Session Management

Handle session lifecycle — creation, resumption, timeout, and cleanup.

TypeScript
class SessionManager {
  private store: Map<string, {
    context: ConversationContext;
    stateMachine: ConversationStateMachine;
    createdAt: number;
    channel: string;
  }> = new Map();

  private readonly sessionTTL = 30 * 60 * 1000; // 30 minutes

  createSession(sessionId: string, channel: string): void {
    this.store.set(sessionId, {
      context: {
        userId: null,
        orderId: null,
        intent: null,
        issueDescription: null,
        turnCount: 0,
        lastActivity: Date.now(),
        slotsFilled: {},
        history: [],
      },
      stateMachine: new ConversationStateMachine(),
      createdAt: Date.now(),
      channel,
    });
  }

  getSession(sessionId: string): ReturnType<typeof this.store.get> {
    const session = this.store.get(sessionId);
    if (!session) return undefined;

    // Check if session is expired
    if (Date.now() - session.context.lastActivity > this.sessionTTL) {
      this.store.delete(sessionId);
      return undefined;
    }

    return session;
  }

  cleanExpiredSessions(): number {
    let cleaned = 0;
    const now = Date.now();
    for (const [id, session] of this.store) {
      if (now - session.context.lastActivity > this.sessionTTL) {
        this.store.delete(id);
        cleaned++;
      }
    }
    return cleaned;
  }
}

Full Example: Customer Support Bot

Putting it all together — a complete customer support bot with state machine, intent routing, slot filling, escalation, and analytics.

TypeScript
class CustomerSupportBot {
  private sessionManager = new SessionManager();
  private analytics = new ChatbotAnalytics();
  private contextStore = new ConversationContextStore();
  private adapters: Map<string, ChannelAdapter> = new Map();

  registerChannel(adapter: ChannelAdapter): void {
    this.adapters.set(adapter.channelName, adapter);
  }

  async handleMessage(
    sessionId: string,
    message: string,
    channel: string
  ): Promise<string> {
    // Get or create session
    let session = this.sessionManager.getSession(sessionId);
    if (!session) {
      this.sessionManager.createSession(sessionId, channel);
      session = this.sessionManager.getSession(sessionId)!;
    }

    const ctx = session.context;
    const startTime = Date.now();

    // Step 1: Check escalation triggers first
    const escalationRule = checkEscalation(ctx, message);
    if (escalationRule) {
      await executeHandoff(sessionId, ctx, escalationRule);
      return "I am transferring you to a human agent now. They will have the full context of our conversation. Please hold on.";
    }

    // Step 2: Detect intent
    const intentResult = await detectIntentWithLLM(message, ctx.history);
    ctx.intent = intentResult.intent;

    // Step 3: Process through state machine
    const response = await session.stateMachine.processInput(message);

    // Step 4: Format for the channel
    const adapter = this.adapters.get(channel);
    const formattedResponse = adapter
      ? adapter.formatMessage(response)
      : response;

    // Step 5: Record metrics
    const responseTime = Date.now() - startTime;
    console.log(`Response time: ${responseTime}ms`);

    return formattedResponse;
  }
}

// Usage
const bot = new CustomerSupportBot();
bot.registerChannel(new WebChatAdapter());
bot.registerChannel(new WhatsAppAdapter());

// Process incoming message
const reply = await bot.handleMessage(
  "session-123",
  "Where is my order ORD-445566?",
  "web"
);
console.log(reply);

Key Takeaways

State machines are the backbone. They give your chatbot predictable, testable conversation flows instead of spaghetti code.
Intent detection bridges the gap. Whether pattern-based or LLM-powered, you need to understand what users want before you can help them.
Context is everything. A bot that forgets prior messages is frustrating. Manage context carefully with session stores and sliding windows.
Slot filling structures data collection. When you need multiple pieces of information, fill slots one at a time through natural conversation.
Escalation is not failure. Knowing when to hand off to a human is a sign of a well-designed bot, not a broken one.
Fallbacks should be tiered. Rephrase, offer options, then escalate — never leave the user stuck in a dead end.
Analytics drive improvement. Track resolution rates, escalation rates, average turns, and top intents to continuously improve your bot.
Multi-channel from day one. Abstract the channel layer so the same bot logic works on web, WhatsApp, Slack, and SMS without duplication.

What Is Next?

In the next lesson, you will learn how to add memory and persistence to your chatbot — storing conversation history in databases, retrieving past interactions, and building bots that remember users across sessions and channels.

Module 13

2/7

🔍 RAG — Retrieval-Augmented Generation

Content Generation Pipeline ✏️

2/7