Human-in-the-Loop | Genkit by Example

This chatbot has an interrupt tool that allows it to ask the user questions with a structured UI. By default, it is set to act like a trivia host (although it's likely to hallucinate some of the answers).

Interrupts allow you to combine automatic tool execution (for e.g. fetching data) with the ability for the LLM to selectively pause the generation loop when out-of-band input is required.

Things to Try

Send a trivia topic like "1990s movies"
Try sending a new chat message instead of using the UI to respond.
Set the system message to something different, like You are an event planner. I'll give you a type of event, then ask me clarifying questions using the askQuestion function to plan a great party. Keep examining past responses to formulate a plan. When you have enough information, generate a detailed party plan in Markdown syntax.

Source Code

// api/route.ts

import { z } from "genkit";
// this example requires beta features
import { genkit } from "genkit/beta";

import { googleAI, gemini20Flash } from "@genkit-ai/googleai";

const ai = genkit({
  plugins: [googleAI()], // set the GOOGLE_API_KEY env variable
  model: gemini20Flash,
});

import genkitEndpoint from "@/lib/genkit-endpoint";
import { AnswerSchema, QuestionSchema } from "../schema";
import { DEFAULT_SYSTEM_MESSAGE } from "../constants";

const askQuestion = ai.defineInterrupt({
  name: "askQuestion",
  description:
    "Use this to directly ask the user a question. The user will see a custom form with the options. The response of this function call will be the user's answer.",
  inputSchema: QuestionSchema,
  outputSchema: AnswerSchema,
});

export const POST = genkitEndpoint(({ system, messages, prompt, resume }) => {
  const chat = ai.chat({
    system: system || DEFAULT_SYSTEM_MESSAGE,
    tools: [askQuestion],
    messages,
  });
  return chat.sendStream({ prompt, resume });
});

// constants.ts

export const DEFAULT_SYSTEM_MESSAGE =
  "You are a trivia game host. The user will provide a subject and you will begin quizzing them. For each question, introduce the question with some fun flavor text, then use the askQuestion tool to ask the question. When they answer a question, tell them whether they're right or wrong then introduce the next question.";

// schema.ts

import { z } from "genkit";

export const QuestionSchema = z.object({
  question: z.string().describe("the text question to display to the user"),
  choices: z
    .array(z.string())
    .describe("choices for a multiple choice question"),
  allowMultiple: z
    .boolean()
    .optional()
    .describe("when true, allows the user to select multiple options"),
  allowCustom: z
    .boolean()
    .optional()
    .describe("when true, allows the user to write-in their own answer"),
});
export type Question = z.infer<typeof QuestionSchema>;

export const AnswerSchema = z.object({
  answer: z.union([z.array(z.string()), z.string()]),
});
export type Answer = z.infer<typeof AnswerSchema>;