-
In the wiki (https://github.com/unslothai/unsloth/wiki) for manually saving to gguf it has the line:
python llama.cpp/convert-hf-to-gguf.py FOLDER --outfile OUTPUT --outtype f16
This should be:
…
-
Hi! :)
I'm using `llama-cpp-python==0.2.60`, installed using this command `CMAKE_ARGS="-DLLAMA_METAL=on" pip install llama-cpp-python`.
I'm able to load a model using `type_k=8` and `type_v=8` …
ddh0 updated
7 months ago
-
It's my first bug report on the GitHub, please, don't be mad if I've done something wrong.
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [☑…
-
Qwen2 Deployment by Ollama fail,prompt"Ollama_llama_server无法找到入口"
Test environment：Ultra 5 125H CPU，Win11 23H2…
-
Hello!
I am trying to install the server package with hipBLAS / ROCm support.
The install fails with a
`cc: error: unrecognized command-line option ‘-Wunreachable-code-break’; did you mean ‘-Wu…
-
### OpenVINO Version
2024.3
### Operating System
Windows System
### Device used for inference
intel UHD Graphics GPU
### Framework
None
### Model used
meta-llama/Llama-3.2-3…
-
**Describe the bug**
I am trying to run the command ilab data generate --taxonomy-path ./taxonomy when I am getting this error
**To Reproduce**
Setup ilab with latest release version on a RHE…
-
llama.cpp now supports grammars:
https://til.simonwillison.net/llms/llama-cpp-python-grammars
Is that something that will come to candle?
It sounds like the approach taken in this python libr…
-
**Describe the bug**
After following the installation instructions for Mac (Apple Metal) on the 'Getting Started' I try to do SDG and see this:
```
$ ilab data generate
INFO 2024-08-22 13:45:45,…
-
Hi,
I see you have built an example for Mistral models that I could build successfully. However, when I try to benchmark such models using GPTSessionBenchmark I get errors like:
`[TensorRT-LLM][ERR…