-
Good day,
Thank you for the help in advance. I have saved a meta model using
```
from prophet.serialize import model_to_json, model_from_json
with open('serialized_model.json', 'w') as fout…
-
During the process of fine-tuning LLama3 using LLama.cpp on my Mac, I encountered this error. I'm a beginner and don't know what caused this issue. I hope an expert can help me.
The model used is: …
-
### What happened?
After last week's updates llama-cli (former main) either chats with itself, outputs random tokens, or stops answering altogether. The problem is the same on CPU and on NVIDIA GPUs…
-
**Describe the bug**
I have tried stable and nighly version - with clean install and JAN fails to load models (standard settings) or fails to run inference on Vulcan (you cn observe model loading bec…
-
### What happened?
There seems to be some kind of memory overlap between contexts created with the same model with the Vulkan backend when the contexts are loaded at the same time.
Freeing the first…
-
### What is the issue?
Hi,
I noticed previous out of memory error fix at version 0.1.45-rc3. [https://github.com/ollama/ollama/issues/5113].
```
ollama run deepseek-coder-v2
```
Now I…
-
### What is the issue?
I get a CUDA out of memory error when sending large prompt (about 20k+ tokens) to Phi-3 Mini 128k model on laptop with Nvidia A2000 4GB RAM. At first about 3.3GB GPU RAM and …
-
https://storage.googleapis.com/deepmind-media/gemma/gemma-2-report.pdf
https://openai.com/index/finding-gpt4s-mistakes-with-gpt-4/
https://github.com/jeasonstudio/chrome-ai
https://ai.meta.com/rese…
-
### Describe the bug
The output window where the llm outputs its text and displays the chat history randomly flickers intermittently.
### Is there an existing issue for this?
- [x] I have sea…
-
### System Info
tgi docker image 2.0.4
### Information
- [X] Docker
- [ ] The CLI directly
### Tasks
- [X] An officially supported command
- [ ] My own modifications
### Reproduction
Passing th…