Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
"prompt_style": { "style_name": "LLAMA3", "system_prompt": "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.", "roles": [ "user", "assistant" ], "intra_message_sep": "\n\n", "inter_message_sep": "<|eot_id|>", "stop": [ "<|end_of_text|>", "<|eot_id|>" ], "stop_token_ids": [ 128001, 128009 ] }
Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
[ ] docker / docker
[X] pip install / 通过 pip install 安装
[ ] installation from source / 从源码安装
Version info / 版本信息
0.17
The command used to start Xinference / 用以启动 xinference 的命令
System Info / 系統信息
请问有关于prompt_style 中几个参数的说明文档吗?
intra_message_sep
和inter_message_sep
这几个参数不明白是什么意思? 由于模型乱回答,重复输出n遍,要么是回答时带着上次的问答文本,想试着调一下prompt。"prompt_style": { "style_name": "LLAMA3", "system_prompt": "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions.", "roles": [ "user", "assistant" ], "intra_message_sep": "\n\n", "inter_message_sep": "<|eot_id|>", "stop": [ "<|end_of_text|>", "<|eot_id|>" ], "stop_token_ids": [ 128001, 128009 ] }
Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?
Version info / 版本信息
0.17
The command used to start Xinference / 用以启动 xinference 的命令
xinference-local -H 启动
Reproduction / 复现过程
调用接口每次回答都有错乱的问题
Expected behavior / 期待表现
不重复,不复述