-
### Before submitting your bug report
- [ ] I believe this is a bug. I'll try to join the [Continue Discord](https://discord.gg/NWtdYexhMs) for questions
- [X] I'm not able to find an [open issue]…
-
Add this https://github.com/nomic-ai/gpt4all as an API.
-
### Describe the bug
For some reason with this model you are only able to load 54 layers to the GPU before it will error out with the following message
```
17:41:28-611506 ERROR Failed to load…
-
Hey, does it work on gpu?
-
Microsoft recently released Phi-3 models in 3 variants (mini, small & medium). Can we add support for this new family of models.
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
I just imported response Synthesizer and tried to use it, I get this error.
my_synthe…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
Hi,
no matter If I use this small snippet, or use my sophisticated Application.…
ghost updated
5 months ago
-
### What happened?
It takes around 7.2min to load a 7b model, which is extremely slow.
see log
###########
# Timings #
###########
mst_eval: 19.77 # ms / token during generation
mst_p_ev…
-
Gemma models that have been quantized using Llamacpp are not working. Please look into the issue
error
"llama.cpp error: 'create_tensor: tensor 'output.weight' not found'"
I will open a issue…
-
**Describe the bug**
The `nitro` process keep using CPU after inference is done.
**Steps to reproduce**
Steps to reproduce the behavior:
1. Install Jan and download the `Mistral Instruct 7B Q4` …
zezic updated
6 months ago