This application can be configured (see config.py) to create your own specialized AI assistant.
Demo: https://bmae.edocloud.be (running the app, the db, and a reverse proxy on an Azure virtual machine)
Requirements: Python 3.10+
$ git clone https://github.com/dodeeric/langchain-ai-assistant-with-hybrid-rag.git
$ cd langchain-ai-assistant-with-hybrid-ragAdd your API keys (only the OpenAI API key is mandatory) and admin password:
$ nano .envOPENAI_API_KEY = "sk-proj-xxx" ==> Go to https://platform.openai.com/api-keys
ANTHROPIC_API_KEY = "sk-ant-xxx" ==> Go to https://console.anthropic.com/settings/keys
LANGCHAIN_API_KEY = "ls__xxx" ==> Go to https://smith.langchain.com (Langsmith)
LANGCHAIN_TRACING_V2 = "true" ==> Set to false if you will not use Langsmith traces
ADMIN_PASSWORD = "xxx" ==> You chose your password
GOOGLE_APPLICATION_CREDENTIALS = "./serviceaccountxxx.json" ==> Path to the Service Account (with VertexAI role) JSON fileConfigure the application:
$ nano config/config.pyInstall required libraries:
$ pip install -U -r requirements.txtLaunch the Chroma DB server: (on the same server as the app or another one)
$ bash db.sh startLaunch the AI Assistant:
$ bash app.sh startRemark: if you get the "streamlit: command not found" error, then log off, then log in, to have the PATH updated.
Go to: http://IP:8080 (the IP is displayed on the screen in the "External URL".)
Go first to the admin interface (introduce the admin password), and scrape some web pages and/or upload some PDF files, then embed them to the vector DB.
For example:
$ sudo apt update
$ sudo apt install nginx
$ sudo nano /etc/nginx/sites-available/streamlitnginxconfserver {
listen 80;
server_name bmae.edocloud.be www.bmae.edocloud.be;
location / {
proxy_pass http://10.0.0.4:8080;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}$ sudo ln -s /etc/nginx/sites-available/streamlitnginxconf /etc/nginx/sites-enabled/streamlitnginxconf
$ sudo systemctl restart nginxGo to: http://172.205.226.216 or http://bmae.edocloud.be
More info: https://ngbala6.medium.com/deploy-streamlit-app-on-nginx-cfa327106050
$ sudo snap install --classic certbot
$ sudo ln -s /snap/bin/certbot /usr/bin/certbot
$ sudo certbot --nginx -d bmae.edocloud.be -d www.bmae.edocloud.be
$ sudo certbot renew --dry-runGo to: https://172.205.226.216 or https://bmae.edocloud.be
From the server on which Chroma is installed:
$ sudo apt install sqlite3
$ cd chromadb
$ sqlite3 chroma.sqlite3sqlite> .tables ===> List of the tables
sqlite> select * from collections; ===> Name of the collection (bmae) & size of the vectors (3072)
sqlite> select count(*) from embeddings; ===> Number of records in the DB
sqlite> select id, key, string_value from embedding_metadata LIMIT 10 OFFSET 0; ===> Display JSON items and PDF pages
sqlite> PRAGMA table_info(embedding_metadata); ===> Structure of the table
sqlite> select * from embedding_metadata where string_value like '%Delper%'; ===> Display matching records
sqlite> select count(*) from embedding_metadata where string_value like '%Delper%'; ===> Display number of matching records
sqlite> .quitFrom the server on which Chroma is installed:
$ git clone https://github.com/flanker/chromadb-admin.git
$ cd chromadb-admin/
$ sudo docker build -t chromadb-admin .
$ sudo docker run -p 3000:3000 chromadb-admin &Configure the "Chroma connection string": Ex.: http://myvm2.edocloud.be:8000
Demo: http://bmae.edocloud.be:3000
Install Ollama:
$ curl -fsSL https://ollama.com/install.sh | shThe Ollama API is now available at 127.0.0.1:11434.
Run the Llama 3 LLM:
$ ollama pull llama3
$ ollama list
$ ollama serveQuery the LLM in a new window:
$ ollama run llama3
>>> What's the capital of France?
>>> /byeTo make Ollama listen on all IPs, not only 127.0.0.1:
$ sudo nano /etc/systemd/system/ollama.service
[Services]
Environment="OLLAMA_HOST=0.0.0.0"
$ sudo systemctl daemon-reload
$ sudo systemctl restart ollama
$ sudo systemctl status ollamaYou can deploy directly from Github repository to Streamlit Community cloud.
Demo: https://bmae-ai-assistant.streamlit.app (running the app)
You can deploy directly from Github repository to Azure Web App service with Github Actions workflow.
Procedure:
A) Create the App service plan and the App service (Web App)
With the Azure Web interface (Console), or with a Bicep or JSON ARM template, or with a Terraform configuration, or with the Azure CLI or Azure Powershell CLI, create:
Example of Bicep ARM template and Azure DevOps pipeline:
File: web-app.bicep
module appServicePlan 'br/public:avm/res/web/serverfarm:0.2.0' = {
name: 'appServicePlanDeployment'
params: {
name: 'ragai-appserviceplan'
skuCapacity: 1
skuName: 'B1' // Standard S is deprecated
kind: 'Linux'
reserved: true // Mandatory if Linux
}
}
module webApp 'br/public:avm/res/web/site:0.3.8' = {
name: 'webAppDeployment'
params: {
kind: 'app,linux'
name: 'ragai-webapp'
serverFarmResourceId: appServicePlan.outputs.resourceId
location: 'westeurope'
siteConfig: {
appSettings: [
{
name: 'WEBSITES_ENABLE_APP_SERVICE_STORAGE'
value: 'false'
}
]
linuxFxVersion: 'PYTHON|3.10'
}
}
}File: azure-pipeline.yml
name: $(Build.DefinitionName)_$(SourceBranchName)_$(Date:yyyyMMdd).$(Rev:r)
variables:
- group: environment-variables
parameters:
- name: location
displayName: The location of the resource group
type: string
default: 'westeurope'
- name: resourcegroup
displayName: The name of the resource group (rg)
type: string
default: 'ragai-rg'
trigger: none
stages:
- stage: Stage_1
displayName: 'Stage 1: Deploy resource group'
jobs:
- job: Job_1
displayName: 'Deploy Job 1'
steps:
- task: AzureCLI@2
displayName: 'Deploy Step 1'
inputs:
azureSubscription: $(subscription)
scriptType: bash
scriptLocation: inlineScript
inlineScript: |
echo "Creating resource group..."
az group create --name ${{parameters.resourcegroup}} --location ${{parameters.location}}
- stage: Stage_2
displayName: 'Stage 2: Deploy web app'
jobs:
- job: Job_1
displayName: 'Deploy Job 1'
steps:
- task: AzureCLI@2
displayName: 'Deploy Step 1'
inputs:
azureSubscription: $(subscription)
scriptType: bash
scriptLocation: inlineScript
inlineScript: |
echo "Deploy web app..."
az deployment group create --resource-group ${{parameters.resourcegroup}} --template-file './azure-web-app/web-app.bicep'B) Configure the deployment source
C) Configure the "Startup Command"
python -m streamlit run Assistant.py --server.port 8000 --server.address 0.0.0.0And click on "Save", then "Continue".
D) Add the "Environment variables"
OPENAI_API_KEY = "sk-proj-xxx" ==> Go to https://platform.openai.com/api-keys
ANTHROPIC_API_KEY = "sk-ant-xxx" ==> Go to https://console.anthropic.com/settings/keys
GOOGLE_API_KEY = "xxx" ==> Go to https://aistudio.google.com/app/apikey
LANGCHAIN_API_KEY = "ls__xxx" ==> Go to https://smith.langchain.com (Langsmith)
LANGCHAIN_TRACING_V2 = "true" ==> Set to false if you will not use Langsmith traces
ADMIN_PASSWORD = "xxx" ==> You chose your password
GOOGLE_APPLICATION_CREDENTIALS = "./serviceaccountxxx.json" ==> Path to the Service Account (with VertexAI role) JSON fileDemo: https://bmae-ragai-webapp.azurewebsites.net (running the app only, not the db)