I'm woking on the docker provided here and I want to run it as an API. I tried the examples also provided README in the devcontainer, and while the fans are spinning well 🤣 and everything seems to be working fine,but I'm not getting any response. I have installed the model and confirmed that it works in the browser. Can you provide any guidance on how to proceed?
root@9ea6b4db2789:~/dalai# ls
alpaca index.js llama package-lock.json server.js venv
config install.js node_modules package.json tmp
root@9ea6b4db2789:~/dalai# node server.js
mkdir /root/dalai
> query: {
url: 'ws://localhost:4000',
model: 'alpaca.7B',
prompt: 'The following is a conversation between a boy and a girl:'
}
{ Core: 'alpaca', Model: '7B' }
exec: /root/dalai/alpaca/main --seed -1 --threads 8 --n_predict 128 --model models/7B/ggml-model-q4_0.bin -p "The following is a conversation between a boy and a girl:" in /root/dalai/alpaca
index.js
const Dalai = require("dalai")
new Dalai().request({
url: "ws://localhost:4000",
model: "alpaca.7B",
prompt: "The following is a conversation between a boy and a girl:",
}, (token) => {
console.log("token", token)
})
server.js
const Dalai = require('dalai')
new Dalai().serve(4000)
I'm woking on the docker provided here and I want to run it as an API. I tried the examples also provided README in the devcontainer, and while the fans are spinning well 🤣 and everything seems to be working fine,but I'm not getting any response. I have installed the model and confirmed that it works in the browser. Can you provide any guidance on how to proceed?
index.js
server.js