Docker compose never responds

[oscadev]

I have ubuntu with a 12GB 3080, 10700k, and 32GB. I installed this by clonging the repo, then doing the few docker commands listed. I have both alpaca 7 and 13 installed. Using the general template, I use this prompt

Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction:
What is 1 plus 1?

### Response:

and it just keeps spinning with the "cancel" button, and it never answers. This is where it stays:

dalai-dalai-1  | exec: /root/dalai/alpaca/main --seed -1 --threads 4 --n_predict 200 --model models/7B/ggml-model-q4_0.bin --top_k 40 --top_p 0.9 --temp 0.8 --repeat_last_n 64 --repeat_penalty 1.3 -p "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
dalai-dalai-1  | 
dalai-dalai-1  | ### Instruction:
dalai-dalai-1  | What is 1 plus 1?
dalai-dalai-1  | 
dalai-dalai-1  | ### Response:
dalai-dalai-1  | " in /root/dalai/alpaca

[DanielNobbe]

Experiencing the same on Ubuntu, not seeing any threads show high usage either

[okellotechconsults]

Same issue, trying to deploy on vps, I tried all methods, started with the npx methods, then docker but they all behave the same way, no response is ever returned, it's a 2CPU ARM64 with 16GB RAM

[cmerther]

Also getting the same on a windows 10 machine with the 13B model.

[fdyhgs]

Tick "debug" on top left and run again. I'd expect an error message to show.

[mirek190]

what do you expect from dead project?

Stop using that ancient dead project and go to llamacpp or koboldcpp ... Also download models from https://huggingface.co/TheBloke ggml versions