-
### What happened?
For llama cpp I had downloaded the q4_k_m quantized [model](https://huggingface.co/jxtngx/Meta-Llama-3.2-1B-Q4_K_M-GGUF/tree/main) and used [llama-bench](https://github.com…
-
The model placed in `extra_model_paths` is not readable, the configuration in `extra_model_paths.yaml` is correct, and I have placed the model in the folder, but the loader is not able to read it, acc…
-
### What is the issue?
It's again the https://github.com/ollama/ollama/issues/6011 issue.
**The issue is with embedding call with the model converted using convert_hf_to_gguf.py.**
litellm.ll…
-
While chaining multiple CNs togehter you will OOM on even two. That is because the way it is built now it needs to load model twice. Make a loader for only the model then split the selection of the mo…
-
from MiDaS.midas.model_loader import default_models, load_model
ModuleNotFoundError: No module named 'MiDaS'
-
### What happened?
Chat template formatting seems to be swapped for Mistral and Llama 2.
Llama2 supports the `` token for system messages, while Mistral simply uses newlines.
Starting llama ser…
-
### What happened?
I am trying to run Qwen2-57B-A14B-instruct, and I used llama-gguf-split to merge the gguf files from [Qwen/Qwen2-57B-A14B-Instruct-GGUF](https://huggingface.co/Qwen/Qwen2-57B-A14B-…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
0.9.0
### Reproduction
opacus使用只要对训练函数使用privacy_engine.make_private函数包裹即可,请问对于sft我该去哪里修改?
model = Ne…
-
I encountered an issue on my Mac where the process stops with the message 'Torch not compiled with CUDA enabled' when using flux-pulid. Do you have any plans to make it compatible with Mac?
-
Hi, I am having an issue with running the sample example in the [quickstart guide](https://github.com/intel-analytics/ipex-llm/blob/main/docs/mddocs/Quickstart/llama_cpp_quickstart.md#3-example-runnin…