search

xusenlinzy / api-for-open-llm

Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, Xverse, SqlCoder, CodeLLaMA, ChatGLM, ChatGLM2, ChatGLM3 etc. 开源大模型的统一后端接口
Apache License 2.0
2.16k stars 252 forks source link

💡 [REQUEST] - <title>vllm支持Qwen1.5-14B-Chat-AWQ和Qwen1.5-14B-Chat-GPTQ-int4吗 #253

Closed dickpy closed 2 months ago

dickpy commented 3 months ago

起始日期 | Start Date

No response

实现PR | Implementation PR

No response

相关Issues | Reference Issues

No response

摘要 | Summary

vllm支持Qwen1.5-14B-Chat-AWQ和Qwen1.5-14B-Chat-GPTQ-int4吗

基本示例 | Basic Example

无

缺陷 | Drawbacks

无

未解决问题 | Unresolved questions

无

xusenlinzy commented 3 months ago

支持的

---- 回复的原邮件 ---- | 发件人 | @.> | | 日期 | 2024年03月27日 17:36 | | 收件人 | @.> | | 抄送至 | @.***> | | 主题 | [xusenlinzy/api-for-open-llm] 💡 [REQUEST] - vllm支持Qwen1.5-14B-Chat-AWQ和Qwen1.5-14B-Chat-GPTQ-int4吗 (Issue #253) |</p> <p>起始日期 | Start Date</p> <p>No response</p> <p>实现PR | Implementation PR</p> <p>No response</p> <p>相关Issues | Reference Issues</p> <p>No response</p> <p>摘要 | Summary</p> <p>vllm支持Qwen1.5-14B-Chat-AWQ和Qwen1.5-14B-Chat-GPTQ-int4吗</p> <p>基本示例 | Basic Example</p> <p>无</p> <p>缺陷 | Drawbacks</p> <p>无</p> <p>未解决问题 | Unresolved questions</p> <p>无</p> <p>— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: <strong><em>@</em></strong>.***></p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/dickpy"><img src="https://avatars.githubusercontent.com/u/38396890?v=4" />dickpy</a> commented <strong> 3 months ago</strong> </div> <div class="markdown-body"> <p>好的谢谢，是不是.env的配置就按照default里面的来，在vllm里面没看到那些模型，还以为不通用</p> </div> </div> <div class="page-bar-simple"> </div> <div class="footer"> <ul class="body"> <li>© <script> document.write(new Date().getFullYear()) </script> Githubissues.</li> <li>Githubissues is a development platform for aggregating issues.</li> </ul> </div> <script src="https://cdn.jsdelivr.net/npm/jquery@3.5.1/dist/jquery.min.js"></script> <script src="/githubissues/assets/js.js"></script> <script src="/githubissues/assets/markdown.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/highlight.min.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/languages/go.min.js"></script> <script> hljs.highlightAll(); </script> </body> </html>