-
### Describe the issue as clearly as possible:
I wrote this as a query on Discord, but I now have it operating as a bug so have made this.
When running the [multiple choices](https://outlines-dev.…
-
打算下载internlm2-7b，但是下载这个模型会出错。下载llama2不会，很奇怪。
执行命令：
python data/hf_dw.py --model internlm/internlm2-7b --use_hf_transfer False
报错：
export HF_ENDPOINT= https://hf-mirror.com
/home/shaoyuantian/…
-
-
### Checklist
- [X] 1. I have searched related issues but cannot get the expected help.
- [X] 2. The bug has not been fixed in the latest version.
- [X] 3. Please note that if the bug-related issue y…
-
当前情况：单卡3090，生成提示词3分钟，出图15秒
盲猜本地llama3-8b表现得速度太慢，是否支持openai这种标准接口？期待回复，再一次感谢作者团队的伟大作品！
-
I try to load pretrained pth of llava: hub/llava-phi-3-mini-pth/model.pth. And I got this strange error:
- used deepspeed zerospeed3 and flash-attn.
```
RuntimeError: Error(s) in loading state_…
-
The intra-application response length is normal
api call response length is missing…
-
This reports mistral.rs as being faster than llama.cpp: https://github.com/EricLBuehler/mistral.rs/discussions/612
But I'm seeing much slower speeds for the same prompt/settings.
Mistral.rs
``…
-
Hi,
Thank you for your great work!
I've been trying to use the Phi-3-Instruct-4B VLM models, but encountered several issues:
- Incorrect LLM backbone choice in phi.py:
https://github.com/R…
-
Hello,
I have an issue with multiple GPU performance.
- I use the recipe `lora_finetune_single_device` with the config `mini_lora_single_device.yaml` on 6000ADA, I got ~5it/s
- I use the recipe `lo…