This guide will help you run the chat application contained in the index.html
file.
Ensure you have the following:
Open the index.html
file and locate the following line:
const chatGPTKey = 'sk-'; // Paste the API key here
Replace 'sk-*****'
with your actual OpenAI GPT-3 API key.
Navigate to the directory containing index.html
and start your local server. For example, if you're using Python's SimpleHTTPServer, you can start it with the command:
python -m SimpleHTTPServer
If you're using Node's http-server, you can start it with the command:
http-server
Open your web browser and navigate to localhost on the port your server is running. For example, if your server is running on port 8000, you would navigate to http://localhost:8000
.
You should now see the chat interface in your browser. You can type messages into the input field and press "Send" to interact with the chatbot.
Please note that this is a simple setup meant for local development and testing. It is not suitable for a production environment.
Create a folder named models
, then download mistral-7b-openorca.Q4_0.gguf
from here https://huggingface.co/Open-Orca/Mistral-7B-OpenOrca and put into the models
folder
Follow the guide here to install llama_cpp Python https://github.com/abetlen/llama-cpp-python
Run the following script to run an OpenAI API server locally. The server should run at port 8000
python3 -m llama_cpp.server --model "./models/mistral-7b-openorca.Q4_0.gguf" --chat_format chatml --n_gpu_layers 1
Open the index.html
file and locate the following line
// Real GPT
// const OPEN_AI_ENDPOINT = 'https://api.openai.com/v1' // Comment this line
// Security, do not deploy this in production
const chatGPTKey = 'sk-*****'; // Create an API key from here
// Local GPT
const OPEN_AI_ENDPOINT = 'http://localhost:8000/v1' // Uncomment this line
Run the application again. It should use localhost for local API inteference.