-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [yes] I am running the latest code. Development is very rapid so there are no tagged versions as …
kmlob updated
9 months ago
-
### ⚠️ 搜索是否存在类似issue
- [X] 我已经搜索过issues和disscussions,没有发现相似issue
### 总结
Ollama本地服务目前支持了Qwen、Gemma、Mistral、Codellama等多种大模型,可参考[Ollama支持的模型](https://ollama.com/library)
这里我在现有的项目中做了如下更改以支持Ollama服务…
-
I followed the instructions, and I was unable to run it under Windows 10 due to `nccl`
-
### Checked other resources
- [X] I added a very descriptive title to this issue.
- [X] I searched the LangChain documentation with the integrated search.
- [X] I used the GitHub search to find a…
-
@leejet @Green-Sky @ggerganov
I do not know cpp and do not have a solid grasp on how ggml works. , but building the repo with cmake -dggml_clblast=ON seems to work as the GPU utilization goes up an…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
在yaml文件中添加dpo_beat:0.1 报错:
[rank5]: File "//.conda/envs/lf/lib/python3.11/site-packages/transformers…
-
lm_eval --model gguf --tasks arc_challenge --num_fewshot 25 --model_args model=codellama,base_url=http://127.0.0.1:8090 --batch_size 16 --log_samples --outp
ut_path ./hzg_llama3_arc_challenge_25shot…
-
Hi,
i am trying to make use of the AWQ quantization to try to load 7B LLama based models onto my RTX 3060 with 12 GB.
This fails OOM for models like https://huggingface.co/TheBloke/leo-hessianai-…
-
When trying to convert Llama-2 models into gguf after fine-tuning, this error occurs:
```
INFO:gguf.vocab:Setting special token type bos to 1
INFO:gguf.vocab:Setting special token type eos to 2
…
-
### System Info
- `transformers` version: 4.38.2
- Platform: Linux-5.15.0-75-generic-x86_64-with-glibc2.35
- Python version: 3.11.8
- Huggingface_hub version: 0.21.4
- Safetensors version: 0.4.…