-
### Reminder
- [X] I have read the README and searched the existing issues.
### Reproduction
运行参数命令如下:
CUDA_VISIBLE_DEVICES=0 API_PORT=8000 python ./src/api_demo.py \
--model_name_or_path sav…
-
### Reminder
- [X] I have read the README and searched the existing issues.
### Reproduction
(base) root@I19c2837ff800901ccf:/hy-tmp/LLaMA-Factory-main/src# CUDA_VISIBLE_DEVICES=0,1,2,3 python3.10 …
-
[WARNING|logging.py:329] 2024-06-14 18:45:29,004 >> Not an error, but Unsloth cannot patch MLP layers with our manual autograd engine since either LoRA adapters
are not enabled or a bias term (like i…
-
### What is the issue?
I am running the qwen:32b model on dual RTX A6000 GPUs (48GB each). There seems to be sufficient VRAM available, with cuda0 using 21GB and cuda1 using less than 5GB. According …
-
**Environment:**
* WSL version: 2.2.4.0
* Kernel version: 5.15.153.1-2
* WSLg version: 1.0.61
* MSRDC version: 1.2.5326
* Direct3D version: 1.611.1-81528511
* DXCore version: 10.0.26091.1-2403…
-
### Your current environment
vllm 0.4.3
CUDA Driver Version: 555.42.02
4060Ti Super * 2
VLLM_ATTENTION_BACKEND=FLASH_ATTN CUDA_VISIBLE_DEVICES=0
python -m vllm.entrypoints.openai.api_server
…
-
### Self Checks
- [X] This is only for bug report, if you would like to ask a question, please head to [Discussions](https://github.com/langgenius/dify/discussions/categories/general).
- [X] I have s…
-
没有模型 qwen-long 呢?字符串的方式指定模型会有各种各样的问题,不知道是不是官方提供的接口标准不一样。qwen-long模型官方价格要低的多,什么时候加上这个模型的支持呢?
https://help.aliyun.com/zh/dashscope/developer-reference/tongyi-thousand-questions-metering-and-billing?spm…
-
### What is the issue?
I installed ollama today, the system is Ubuntu2204,I downloaded llama3.1-405b-Q2.gguf,There are 9 split files in total. Ollama create llama -f Modelfile.txt is completed succes…
-
**例行检查**
[//]: # (方框内删除已有的空格,填 x 号)
+ [ ] 我已确认目前没有类似 issue
+ [ ] 我已确认我已升级到最新版本
+ [ ] 我已完整查看过项目 README,尤其是常见问题部分
+ [ ] 我理解并愿意跟进此 issue,协助测试和提供反馈
+ [ ] 我理解并认可上述内容,并理解项目维护者精力有限,**不遵循规则的 issue 可能…