janhq / cortex.cpp

Run and customize Local LLMs.
https://cortex.so
Apache License 2.0
1.92k stars 107 forks source link

bug(flaky): "Model not found" after user selects a model variant #1287

Open 0xSage opened 6 hours ago

0xSage commented 6 hours ago

Cortex version

v75

Describe the Bug

This issue happens on Windows, the 1st time users try to pull each model. Meaning, it happens many times, each time user is trying to pull a new* model.

But, retrying the same request a second time works just fine.

(base) PS C:\Users\n\cortexcpp-nightly\models> cortex-nightly.exe pull llama3.1
Select an option
1. 8b-onnx
2. 8b-gguf
3. onnx
4. gguf

Select an option (1-4): 1
Model not found

After retrying:

(base) PS C:\Users\n\cortexcpp-nightly\models> cortex-nightly.exe pull llama3.1
Select an option
1. 8b-onnx
2. 8b-gguf
3. onnx
4. gguf

Select an option (1-4): 1
Validating download items, please wait..
...

Progresses as normal

Steps to Reproduce

No response

Screenshots / Logs

No response

What is your OS?

What engine are you running?

0xSage commented 5 hours ago
(base) PS C:\Users\n\cortexcpp-nightly\models\llama3.1> cortex-nightly.exe pull mistral
Select an option
1. 7b-tensorrt-llm-linux-ada
2. tensorrt-llm-linux-ada
3. 7b-tensorrt-llm-linux-ampere
4. tensorrt-llm-linux-ampere
5. 7b-tensorrt-llm-windows-ada
6. 7b-tensorrt-llm-windows-ampere
7. tensorrt-llm-windows-ampere
8. tensorrt-llm-windows-ada
9. onnx
10. gguf
11. 7b-onnx
12. 7b-gguf

Select an option (1-12): 10
Model not found