-
### 是否已有关于该错误的issue或讨论? | Is there an existing issue / discussion for this?
- [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions
### 该问题是否在FAQ中有解答? | Is there an existing…
-
大佬们请问一下,qwen1.5是不是不使用flash-attention加速推理。那为什么相同硬件、安装包的两台服务器,模型运行速度不一样?Token都差不了多少
-
how to train Qwen1.5-MoE-A2.7B in 8 A100 GPU?
-
### 是否已有关于该错误的issue或讨论? | Is there an existing issue / discussion for this?
- [X] 我已经搜索过已有的issues和讨论 | I have searched the existing issues / discussions
### 该问题是否在FAQ中有解答? | Is there an existing ans…
-
**问题描述 / Problem Description**
Prompt config中的修改没有效果,目前已测试GLM3-6B, QWEN1.5 13B。
![d3334f1e4b568284983a19e961fd0d0](https://github.com/chatchat-space/Langchain-Chatchat/assets/125440255/04a3d912-12…
-
![2](https://github.com/QwenLM/Qwen-Agent/assets/94165844/37f1f1b4-8b50-4834-8345-cae087d85686)
请问这个怎么解决
-
### What happened?
1、First, convert the Qwen 72B chat model to the gguf format model. Then, when using the Qwen 72B chat model for inference in llamcpp with the prompt "Please summarize the character…
-
Follow guide: https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/llama_cpp_quickstart.html
command:
`main.exe -m "C:\Users\admin\Desktop\ipex\llama-cpp\llama-cpp\libs\models\qwen1_5-1_8b-…
-
qwen自带推理:
第一身份准确率:0.968421052631579
第二身份准确率:0.9368421052631579
一条数据平均耗时:0.9585727064233077
slang推理:
第一身份准确率:0.6631578947368421
第二身份准确率:0.7894736842105263
一条数据平均耗时:0.5773725032806396
-
机器是8卡4090.用vllm部署32b-chat没问题,就是慢,部署32bAWQ-chat后,输入一个你好都要很久,而且出来一堆乱码。模型加载就是用的llm = LLM(model="Qwen/Qwen1.5-7B-Chat-AWQ", quantization="awq")