How I created a custom GPT-based chatbot in less than 10 minutes with LlamaIndex | by Aashish Nair | April 2023

Presentation and implementation with Python

Not long ago I read an article by Jerry Liu that introduced LlamaIndex, an interface that uses GPT to synthesize query responses using user-supplied information.

It immediately caught my eye and I knew I had to try it for myself.

After all, many of the great language models (LLMs) that have taken the world by storm have limited use cases because they are not trained with the data available to their users.

Thus, there is always a demand for customizable LLMs tailored to the needs of their users. Simply put, companies need LLMs that perform tasks such as text summarization, question and answer (Q&A), and text generation with their own information.

To see if LlamaIndex has the potential to meet this demand, I played around with its features and was really surprised at what I was able to do. I even ended up building a Q&A chatbot in just about 10 minutes!

Here, I accompany you on my journey.

What does LlamaIndex do?

LlamaIndex generates query responses by connecting LLMs to information provided by users.

As detailed in The documentationUsing LlamaIndex involves the following steps:

1. Load into documents
2. Analyze documents into nodes (optional)
3. Build the index
4. Build indices on top of constructed indices (optional)
5. Query index

Essentially, LlamaIndex loads your data into a document object and then converts it to an index. When the index receives a query, it forwards the query to a GPT prompt to synthesize a response. By default, this is done with OpenAI text-davinci-003 model.

Although this process looks quite complicated, it can be done with very little code, as you are about to find out.

Facility

To test the versatility of LlamaIndex, I ended up building 3 different chatbots, each bot being built with a different data source. For the sake of brevity, the previously mentioned optional steps (i.e. steps 2 and 4) will be omitted.

First, let’s take care of the prerequisites.

LlamaIndex and OpenAI can be installed from pip using the following commands:

pip install llama-index
pip install openai

Users will also need an API key from OpenAI:

import os
os.environ('OPENAI_API_KEY') = 'API_KEY'

The project will also need the following imports:

Loading data

Data can be loaded manually or via a data loader. For this case, I loaded 3 types of data:

The first index will be created with the local .txt file, which is in a folder named data. This data will be loaded manually.

Build the clues

With all the data loaded into document objects, we can build the index for each chatbot.

Each index can be built from the document object with one line.

Query index

Built indexes can now generate responses for any given query. Again, this step can be completed with a one-liner.

1. Query index build with .txt file

In case you were wondering, this is the correct answer.

2. Query the index built with the Wikipedia page (theme: apples)

3. Query the index built with YouTube video (subject: vanilla cake recipe)

Finally, it should also be noted that indexes will only provide answers to queries when they contain the necessary context.

Here’s how the same index created with data from the YouTube video would answer a completely irrelevant query.

Fortunately, it seems that LlamaIndex has taken action against hallucinations (i.e. when a model confidently gives an answer that is not justified by the given data).

Deploy Chatbots with a Web App

Finally, we can create a web application in order to share the built indices with end users.

To do this, we first need to register the indices using the save_to_disk method.

These indices will be used in a Streamlit application. The underlying code for the entire application is as follows:

In the app, users can select the data source they want to base their questions on and write their query in the box provided.

We can see how indexes work after running the application:

streamlit run app.py

Querying the index built with the .txt file (my favorite fruits):

Querying the index built with the Wikipedia page (apples):

Querying the index built with the Youtube video (vanilla cake recipe):

Pretty cool, right? We created a functional web application in just 10 minutes!

Final remarks

So far I have only implemented the basic functionality of the LlamaIndex interface. Many areas were not explored in this project, such as customizing LLMs and using non-default settings.

For more information, I invite you to visit The documentation.

If you’re considering experimenting with this tool yourself, beware of the cost of using OpenAI’s API. This project only cost me a dollar, but that can be attributed to my work with small documents (price is based on how many tokens you use). If you get too carried away, you might end up with a bad bill.

Finally, all the source code used to create the Q&A chatbots is accessible in this Github repository:

Thanks for the reading!