search

xusenlinzy / api-for-open-llm

Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, Xverse, SqlCoder, CodeLLaMA, ChatGLM, ChatGLM2, ChatGLM3 etc. 开源大模型的统一后端接口
Apache License 2.0
2.16k stars 252 forks source link

💡 [REQUEST] - 请求支持CodeLlama-70b-Instruct-hf #255

Closed Reset816 closed 2 months ago

Reset816 commented 3 months ago

起始日期 | Start Date

无

实现PR | Implementation PR

无

相关Issues | Reference Issues

无

摘要 | Summary

无

基本示例 | Basic Example

无

缺陷 | Drawbacks

无

未解决问题 | Unresolved questions

Chat use: The 70B Instruct model uses a [different prompt template](https://huggingface.co/codellama/CodeLlama-70b-Instruct-hf#chat_prompt) than the smaller versions. To use it with transformers, we recommend you use the built-in chat template:

70B的 prompt template 和现在已经支持的7/13/34B不一样。不知道能否直接用