-
### The model to consider.
https://huggingface.co/serpdotai/sparsetral-16x7B-v2-SPIN_iter1
https://huggingface.co/LoneStriker/sparsetral-16x7B-v2-8.0bpw-h8-exl2/tree/main
https://huggingface.co/h…
-
having the ability to use the api to paid services is cute and all.
can we have local only.
nobody wants to pay for these services anymore especially as llama3.1 blew them away with costly tie…
-
Hello,
I've tried running realtoxicityprompts (github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/tasks/realtoxicityprompts/) through the Hugging Face leaderboard backend code (https://h…
-
### Checklist
- [ ] 1. I have searched related issues but cannot get the expected help.
- [ ] 2. The bug has not been fixed in the latest version.
- [ ] 3. Please note that if the bug-related issue y…
-
### What is the issue?
Ollama using Docker mode.
When execute 'sudo docker exec -it ollama ollama run nemotron:latest',
or "sudo docker exec -it ollama ollama run qwen2.5:72b"
it replied "GGGGGGG…
-
''ipex-llm[cpp]==2.5.0b20240527 is consistent with [v0.1.34] of ollama.
Our current version is consistent with [v0.1.39] of ollama.''
Is it possible to update supported ollama version to 0.3.x?
-
### System Info
NVIDIA-SMI 535.154.05
Driver Version: 535.154.05
CUDA Version: 12.4
- GPU properties
- GPU name: NVIDIA L20
- GPU memory size: 46068MiB
- Libraries
- Te…
-
### Your current environment
```text
The output of `python collect_env.py`
```
### 🐛 Describe the bug
https://github.com/hiyouga/LLaMA-Factory/issues/4049
transformers+lora
### 你是否已经阅读并同意《Datawhale开源项目行为准则》？
- [X] 我已阅读并同意[《Datawhale开源项目行为准则》](h…
-
### 🚀 The feature, motivation and pitch
Using Qwen2.5 model : ValueError: This model does not support the 'embedding' task. Supported tasks: {'generate'}
reproduction :
`python -m vllm.entryp…