Build a RAG chatbot with Astro, Postgres, and LlamaIndex
A step-by-step guide for building a RAG chatbot in an Astro application with LlamaIndex and Postgres
Prerequisites
To follow the steps in this guide, you will need the following:
- Node.js 18 or later
- A Neon account
- An OpenAI account
- An AWS account
Steps
- Generate the OpenAI API token
- Provisioning a Serverless Postgres
- Create a new Astro application
- Setting up a Postgres database connection
- Define the Astro application routes
- Dockerize your Astro application
- Deploy your Astro application to Amazon ECS
- Configure GitHub Actions for Continuous Deployment (CD) Workflow
Generate the OpenAI API token
To create vector embeddings, you will use OpenAI API with LlamaIndex. To set up OpenAI, do the following:
- Log in to your OpenAI account.
- Navigate to the API Keys page.
- Enter a name for your token and click the Create new secret key button to generate a new key.
- Copy and securely store this token for later use as the OPENAI_API_KEY environment variable.
Provisioning a Serverless Postgres
Using a serverless Postgres database lets you scale compute resources down to zero, which helps you save on compute costs.
To get started, go to the Neon Console and create a project.
You will then be presented with a dialog that provides a connection string of your database. Click on the Pooled connection option and the connection string automatically updates.
All Neon connection strings have the following format:
user
is the database user.password
is the database user’s password.endpoint_hostname
is the host withneon.tech
as the top-level domain (TLD).port
is the Neon port number. The default port number is 5432.dbname
is the name of the database.neondb
is the default database created with a Neon project if you do not define your own database.?sslmode=require
an optional query parameter that enforces SSL mode for better security when connecting to the Postgres instance.
Save the connection string somewhere safe. It will be used to set the the POSTGRES_URL variable later.
Create a new Astro application
Let’s get started by creating a new Astro project. Open your terminal and run the following command:
npm create astro
is the recommended way to scaffold an Astro project quickly.
When prompted, choose:
Empty
when asked how to start the new project.Yes
when asked if you plan to write Typescript.Strict
when asked how strict Typescript should be.Yes
when prompted to install dependencies.Yes
when prompted to initialize a git repository.
Once that’s done, change the project directory and start the app:
The app should be running on localhost:4321. Let's close the development server for now.
Next, execute the command in your terminal window below to install the necessary libraries and packages for building the application:
The command installs the following packages:
dotenv
: A library for handling environment variables.ai
: A library to build AI-powered streaming text and chat UIs.llamaindex
: A data framework for creating LLM applications.
Next, make the following additions in your astro.config.mjs
file to populate the environment variables and make them accessible via process.env
object:
Then, add the following code to your tsconfig.json
file to make relative imports within the project easier:
Let's move on to integrating Tailwind CSS into the Astro application.
Add Tailwind CSS to the application
For styling the app, you will be using Tailwind CSS. Install and set up Tailwind at the root of our project's directory by running:
When prompted, choose:
Yes
when prompted to install the Tailwind dependencies.Yes
when prompted to generate a minimaltailwind.config.mjs
file.Yes
when prompted to make changes to Astro configuration file.
After making the selections outlined above, the command finishes integrating TailwindCSS into your Astro project and installs the following dependencies:
tailwindcss
: TailwindCSS as a package to scan your project files and generate corresponding styles.@astrojs/tailwind
: An adapter that brings Tailwind's utility CSS classes to every.astro
file and framework component in your project.
Let's move on to integrating React into the Astro application.
Integrate React in your Astro project
To prototype the reactive user interface quickly, you will use React as the library with Astro. In your terminal window, execute the following command:
npx
allows us to execute npm package binaries without having to install npm
globally.
When prompted, choose the following:
Yes
to install the React dependencies.Yes
to make changes to Astro configuration file.Yes
to make changes totsconfig.json
file.
Let's move on to enabling server-side rendering in the Astro application.
Enabling Server Side Rendering in Astro using Node.js Adapter
To interact with the chatbot over a server-side API, you will enable server-side rendering in your Astro application. Execute the following command in your terminal:
When prompted, choose:
Yes
to install the Node.js dependencies.Yes
to make changes to Astro configuration file.
After making the selections outlined above, the command finishes integrating the Node.js adapter into your Astro project and installs the following dependency:
@astrojs/node
: The adapter that allows your Astro SSR site to deploy to Node targets.
Let's move on to loading the Postgres URL through an environment variable in the Astro application.
Setting up a Postgres database connection
Create an .env
file in the root directory of your project with the following environment variable to initiate the setup of a database connection:
The file, .env
, should be kept secret and not included in your Git history. Ensure that .env
is added to the .gitignore
file in your project.
Define the Astro application routes
The structure below is what our src/pages
directory will look like at the end of this section:
index.astro
will serve responses with dynamically created HTML to incoming requests at the index route.api/chat.ts
will serve responses as an API Endpoint to incoming requests at/api/chat
.api/learn.ts
will serve responses as an API Endpoint to incoming requests at/api/learn
.
Build Conversation User Interface using Vercel AI SDK
Inside the src
directory, create a Chat.jsx
file with the following code:
The code above does the following:
- Imports the
useChat
hook byai
SDK to manage the conversation between the user and the chatbot. It simplifies the management of the conversation between the user and the chatbot. By default, it posts to the/api/chat
endpoint to obtain responses from the chatbot. - Exports a React component that returns a form containing an
<input>
element to allow users to enter their query. - Creates a conversation UI looping over the set of messages (managed by the AI SDK).
Now, let’s create a component that will allow the user to add text to the chatbot's knowledge.
Build UI to update Chabot’s Knowledge
Inside the src
directory, create a Learn.jsx
file with the following code:
The code above does the following:
- Imports
useState
hook by React. - Exports a React component that returns a form containing a
<textarea>
element to accept a string. - Upon form submission, it posts the message string input by the user as JSON to the
/api/learn
endpoint.
Build an entrypoint React component
Inside the src
directory, create a App.jsx
file with the following code:
The code above imports and renders both the Chat and the Learn component created earlier. Finally, update the index.astro
file to import the App
component:
The changes above import the App component. Additionally, using Astro's client:load
directive the code makes sure that the React application is hydrated immediately on the page.
Let's move on to using Postgres as the vector store for your chatbot.
Initialize Postgres Vector Store in LlamaIndex
To query and add documents to the Postgres vector store, you are going to use PGVectorStore
class to communicate. Inside the src
directory, create vectorStore.ts
with the following code:
The code above begins with importing the dotenv/config
, loading all the environment variables into the scope. Additionally, it exports an instance of PGVectorStore
initialized using the Postgres pooled connection URL obtained earlier.
Let's move on to building the chat API endpoint.
Build the Chat API Endpoint
The Vercel AI SDK uses /api/chat
by default to obtain the chatbot responses. Create a file src/pages/api/chat.ts
with the following code:
The code above does the following:
- Imports the vector store instance that is using Postgres.
- Imports the VectorStoreIndex helper by llamaindex.
- Exports a POST HTTP Handler which responds to incoming POST requests on
/api/chat
. - Destructs messages array from the request body.
- Creates the LlamaIndex's query engine using Postgres as the vector store.
- Creates a stream handler that streams the response from LlamaIndex's query engine.
- Returns the stream handler as a standard Web Response.
Let's move on to building the endpoint to update chatbot's knowledge.
Build the Learn API Endpoint
As you saw earlier, with LlamaIndex you do not need to manually create context and pass it to an external API. The vector store is searched for similar vector embeddings based on the user query, internally. To keep the knowledge of the chatbot up-to-date, create a file src/pages/api/learn.ts
with the following code:
The code above does the following:
- Imports the vector store instance that is using Postgres.
- Imports the VectorStoreIndex and storageContextFromDefaults helpers by llamaindex.
- Exports a POST HTTP Handler which responds to incoming POST requests on
/api/learn
. - Destructs the text message from the request body.
- Creates a LlamaIndex document with text as its sole data.
- Pushes the vector embeddings generated for the text along with the document metadata to the Postgres database.
Let's move on to dockerizing the Astro application.
Dockerize your Astro application
To dockerize your Astro application, you are going to create two files at the root of your Astro project:
.dockerignore
: The set of files that would not be included in your Docker image.Dockerfile
: The set of instructions that would be executed while your Docker image builds.
Create the .dockerignore
file at the root of your Astro project with the following code:
Create the Dockerfile
file at the root of your Astro project with the following code:
The Dockerfile above defines the following set of actions:
- Sets up Node.js 20.11.0.
- Sets the environment to
production
withNODE_ENV
environment variable. - Instasll the dependencies of your Astro project.
- Builds the application with
astro build
. - Sets the
PORT
environment variable to80
(default port on Amazon ECS). - Sets the
HOST
environment variable to0.0.0.0
to listen to all incoming requests on the host. - Runs the production server with
node ./dist/server/entry.mjs
command.
Now, let's create a file that creates Amazon ECS Task definition during the deployment via GitHub Actions. This is useful as it protects the secrets stored in the GitHub repo. Create an env.mjs
file at the root of your Astro application with the following code:
The code above does the following:
- Imports
dotenv/config
to load all the environment variables into the scope and make them accessible via theprocess.env
object. - Validates the presense of
AWS_ACCOUNT_ID
,POSTGRES_URL
, andOPENAI_API_KEY
environment variables. - Writes a
task-definition.json
file at the root of your Astro application following the format used earlier, and adds theOPENAI_API_KEY
andPOSTGRES_URL
environment variables.
Deploy your Astro application to Amazon ECS
In this section, you will learn how to create an Amazon ECR repository for your Docker-based deployments, spin up Amazon ECS Cluster and an ECS service, create AWS ECS Task Definition(s) and grant ECS Full Access to your AWS IAM user.
Create Amazon ECR private repository
- Open the Amazon ECR console, and click Get started.
- Enter a repository name, say
astro-repo
, for example. Scroll down and choose Create repository.
You are now done setting up an Amazon ECR repository. Let’s move on to configuring IAM roles for your account.
Configure your IAM Roles
- Open the IAM console, and click Create role.
- Select AWS Service and choose Elastic Container Service as the Service or use case.
- Filter the large set of permissions policies, select AmazonECS_FullAccess only, and click Next.
- Enter
ecsTaskRole
as the Role name.
- Go back to the IAM Console, and click Create role.
- Select AWS Service and choose Elastic Container Service Task as the Service or use case.
- Filter the large set of permissions policies, select AmazonECSTaskExecutionRolePolicy only, and click Next.
- Enter
ecsTaskExecutionRole
as the Role name.
You are now done setting up IAM Roles for your account. Let’s move on to creating an Amazon ECS task definition.
Create an Amazon ECS Task Definition
- Open the Amazon ECS Console and choose Task Definitions. Further, select Create new task definition with JSON.
- Copy the following JSON in the field, and click Create.
You are now done setting up an Amazon ECS task definition for your service. Let's move on to creating an Amazon ECS Cluster.
Create an Amazon ECS Cluster
- Open Amazon ECS Console and click Create cluster.
- Enter a name for your cluster, say
astro-cluster
, and choose Create.
You are now done setting up an Amazon ECS Cluster for your service.
Let's move on to creating an Amazon ECS Service.
Create an Amazon ECS Service
- Click on the Cluster created in the section earlier, and click on Create in the Services section.
- Enter a name for your service, say
astro-service
, and expand the Networking section.
- Select the VPC created earlier (or the default one). Select Create a new security group option and enter the following details for it:
- Security group name:
astro-sg
- Security group description:
astro sg
- Inbound rules for security groups:
- Type:
HTTP
- Source:
Anywhere
- Type:
- Security group name:
You are now done creating an ECS Service in your ECS Cluster. Let's move on to creating access keys for IAM users for your account.
Create Access Keys for IAM users
- In the navigation bar in your AWS account, choose your user name, and then choose Security credentials.
- Scroll down to Access keys and click on Create access key.
- Again, click on Create access key.
- Copy the Access key and Secret access key generated to be used as
AWS_ACCESS_KEY_ID
andAWS_ACCESS_KEY_SECRET
respectively.
Let's move on to configuring GitHub Workflows for continuous deployments.
Configure GitHub Actions for Continuous Deployment (CD) Workflow
First, let's add all the required environment variables obtained in the AWS steps above to your GitHub repo as repository Secrets. Go to your GitHub repository's Settings, and click on Secrets and Variables. Then, click on New repository secret.
- Enter AWS_ACCOUNT_ID as the value obtained earlier.
- Enter AWS_ACCESS_KEY_ID as the value obtained earlier.
- Enter AWS_ACCESS_KEY_SECRET as the value obtained earlier.
- Enter POSTGRES_URL as the value obtained earlier.
- Enter OPENAI_API_KEY as the value obtained earlier.
Next, to automate deployments of your Astro application, you are going to use GitHub Actions. Create a .github/workflows/deploy.yml
with the following code:
The workflow above does the following:
- Allows itself to be triggered manually or when a git push is done to the master branch.
- Sets global environment variables as per your AWS setup variables (we obtained earlier during the setup).
- Loads the environment variables added to the GitHub repo as secrets into the scope.
- Writes the task definition including the environment variables.
- Builds and pushes the Docker image to Amazon ECR.
- Loads the updated (if) task definition to Amazon ECS.
Now, push the added GitHub workflow file to your GitHub repo. Follow the steps below to trigger the deployment:
-
Go to your GitHub repository's Actions tab.
-
Select Deploy to Amazon ECS workflow.
-
Click Run workflow.
-
Once the action is completed, open ECS Console and select your service.
- Click on the Tasks tab.
- Click on the completed Task.
- Click open address to open your deployment.
Summary
In this guide, you learned how to build a RAG Chatbot using LlamaIndex, Astro, and Neon Postgres. Additionally, you learned how to automate deployments of your Astro application using GitHub Actions to Amazon ECS on Amazon Fargate.
Need help?
Join our Discord Server to ask questions or see what others are doing with Neon. Users on paid plans can open a support ticket from the console. For more details, see Getting Support.