Building a chatbot with OpenAI, Vercel AI and Xata

In today's data-driven world, efficient interaction with databases is a crucial aspect of many applications. But what if we could go beyond conventional search methods and enable a natural language conversation with our databases?

At Xata, we aim to provide developers with the tools to build powerful applications that can interact with data in a natural way. Built-in with our core APIs and SDKs, we offer a powerful ask endpoint that allows you to ask questions about your data and get the answers that matter most.

However, how would you integrate Xata with an existing application built with OpenAI? In this tutorial, we'll show you how to integrate OpenAI's function calling feature with Xata's TypeScript SDK to create a chatbot that can use search to answer questions about your data.

In a previous post, we introduced the concept of using our built-in ask endpoint to simplify the process of querying your data.

Before we begin, make sure you have the following prerequisites:

• Existing project configured with the Xata CLI
• Xata API key
• OpenAI API key

In your preferred serverless environment, make sure you install the OpenAI API Library and Vercel AI library to get started.

After ensuring your prerequisites are met, you can integrate Xata with your existing OpenAI application in three steps: Define a search function for AI, ask questions about your data, and run completions while streaming the results.

First, we'll define a function that allows us to search our database using OpenAI's function calling feature.

const functions: CompletionCreateParams.Function[] = [
  {
    name: 'full_text_search',
    description: 'Full text search on a branch',
    parameters: {
      type: 'object',
      properties: {
        query: {
          type: 'string',
          description: 'The search query'
        }
      },
      required: ['query']
    }
  }
];

If we want to allow OpenAI to fine-tune the search results, we can add more options to the parameters object. For example, we can add a fuzziness parameter to allow for fuzzy search.

const functions: CompletionCreateParams.Function[] = [
  {
    name: 'full_text_search',
    description: 'Full text search on a branch',
    parameters: {
      type: 'object',
      properties: {
        query: {
          type: 'string',
          description: 'The search query'
        },
        fuzziness: {
          type: 'number',
          description: 'Maximum levenshtein distance for fuzzy search, minimum 0, maximum 2'
        }
      },
      required: ['query']
    }
  }
];

Now that we have our search function defined, we can use the OpenAI library to ask a question about our data.

const openai = new OpenAI({
  // Make sure to properly load and set your OpenAI API key here
  apiKey: process.env.OPENAI_API_KEY
});
 
const model = 'gpt-3.5-turbo';
 
const response = await openai.chat.completions.create({
  model,
  messages: [
    {
      role: 'user',
      content: question
    }
  ],
  functions
});

With the functions defined, the AI will be able to call the full_text_search function and pass the query parameter with the parts of the question that are relevant to the search.

To enhance the results, we can include additional information in the messages array as system messages. For instance, we can provide instructions to the AI or offer hints related to our database.

const { schema } = await api.branches.getBranchDetails({ workspace, region, database, branch });
 
const response = await openai.chat.completions.create({
  model,
  stream: true,
  messages: [
    {
      role: 'user',
      content: question
    },
    {
      role: 'system',
      content: `
        Workspace: ${workspace}
        Region: ${region}
        Database: ${database}
        Branch: ${branch}
        Schema: ${JSON.stringify(schema)}
 
        Reply to the user about the data in the database, do not reply about other topics.
        Only use the functions you have been provided with, and use them in the way they are documented.
      `
    }
  ],
  functions
});

OpenAI provides a variety of models, which you can find listed here. If you require a different model that aligns more closely with your specific use case, you can easily switch the model parameter.

Finally, we can run the completion and stream the results to the client.

const stream = OpenAIStream(response, {
  experimental_onFunctionCall: async ({ name, arguments: args }, createFunctionCallMessages) => {
    switch (name) {
      case 'full_text_search': {
        const response = await api.searchAndFilter.searchBranch({
          workspace,
          region,
          database,
          branch,
          query: args.query as string,
          fuzziness: args.fuzziness as number
        });
 
        const newMessages = createFunctionCallMessages(response);
 
        return openai.chat.completions.create({
          messages: [...messages, ...newMessages],
          stream: true,
          model,
          functions
        });
      }
      default:
        throw new Error('Unknown OpenAI function call name');
    }
  }
});
 
return new StreamingTextResponse(stream);

We have chosen to stream the results to the client, but you can also wait for the completion to finish and return the results as a single response.

With React and the useChat hook you can easily create a chatbot that can answer questions about your data.

const { messages, append, reload, stop, isLoading } = useChat({
  // The route to the endpoint we have just created
  api: '/api/chat',
  initialMessages
});

Congratulations! You've just built a powerful system that combines the capabilities of OpenAI, Vercel AI and Xata's database API. Users can now engage in natural language conversations with their databases, and OpenAI will utilize the provided functions to perform searches and retrieve relevant information.

By following this tutorial, you've learned how to integrate multiple APIs, handle requests and create interactive responses. This foundation can be extended to create even more sophisticated systems that enable seamless human-machine interactions with data.

If you want to learn more about Xata's database API, check out our documentation and come say hi on our Discord if you have any questions.

Happy coding! 🦋