-
Hey there! Super cool project. Thought I'd add some of the (yet to be documented) steps that I took to get the application working on my macbook pro with an M1 chip.
I did not use the docker image …
-
Hello :)
I have an issue with this app when it comes to connection to ooba with openai api.
I changed port to 5000 in text-default.json config file to match port with that from ooba. Bot turns on j…
-
I'm having an issue where I'm trying to run an example using a zero shot agent and a basic tool via your short_instruction example.
If I load in the OpenAI api as the LLM and run all the other co…
-
### System Info
jetson-agix-orin
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially supported task …
-
We tried many local models like LLAMA, VICUNA, OPENASSIST, GPT4ALL in their 7b versions. None seem to give results like the CHATGPT API.
we would like to try to test new models, which can be loade…
-
Exllama v2 crashes when starting to load in the third gpu. No matter if the order is 3090,3090,A4000 or A4000,3090,3090, when I try to load the turboderp Mistral Large 2407 exl2 3.0bpw it crashes af…
-
### What happened?
I am trying to run a Q4_0_4_4 quantized Llama3 8B model. This is my config :
```
/home/piuser/Desktop/Abhrant/llama-cpp-BLAS/llama.cpp/llama-cli -m /home/piuser/Desktop/Abhran…
-
### Describe the bug
While investigating another issue I noticed that the LLaMA 3 Instruct template is not being tokenized correctly when using the Oobabooga llama.cpp loader, see https://github.co…
-
I'm seeing an example of 29s of audio rendered in ~3s, so about a 10:1 ratio on a 4090 here:
https://github.com/RandomInternetPreson/text_generation_webui_xtt_Alts/tree/main#example
But on my 40…
-
It would be fantastic to have the possibility of running it on local Language Model implementations such as Vicuna or other open-source alternatives.