issues
search
xusenlinzy
/
api-for-open-llm
Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, Xverse, SqlCoder, CodeLLaMA, ChatGLM, ChatGLM2, ChatGLM3 etc. 开源大模型的统一后端接口
Apache License 2.0
2.16k
stars
252
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
TypeError: 'NoneType' object is not subscriptable
#243
deauss2017
closed
2 months ago
0
docker vllm运行qwen1.5-7b-chat无法部署,报错:Fatal Python error: Bus error
#242
syusama
opened
4 months ago
2
Qwen1.5推理报错RuntimeError: cannot reshape tensor of 0 elements into shape [-1, 0] because the unspecified dimension size -1 can be any value and is ambiguous
#241
syusama
opened
4 months ago
6
Qwen1.5 提示模板缺少默认 system message
#240
liuyanyi
closed
4 months ago
1
34b 模型 ,用int4 ,vllm进行推理,单张24G 4090GPU,显示显存不足
#239
haohuisss
closed
2 months ago
10
Qwen2运行报错
#238
yinggoga
closed
4 months ago
1
fix for max_tokens larger than max_model_len using vllm engine
#237
lzhfe
closed
4 months ago
0
使用 vllm 推理,在文本末尾缺失信息
#236
parasol-ry
closed
2 months ago
10
Qwen1.5-7B-Chat 使用API呼叫 completions 功能未能生成下文
#235
kanslor
closed
2 months ago
2
Update the version of transformer in requirements.txt
#234
Tendo33
closed
4 months ago
0
💡 [REQUEST] - <能否实现一个功能,就是api一段时间无调用的时候,自动卸载到cpu甚至硬盘>
#233
Tongjilibo
closed
3 weeks ago
0
Fix vllm stream function call
#232
lzhfe
closed
4 months ago
0
vllm模式function call流式返回异常
#231
lzhfe
closed
4 months ago
1
Internlm2-chat-7b无法生成正确的回复
#230
AlanLu0808
closed
2 months ago
2
internlm2-chat-20b,成功运行后,收到请求无回复
#229
deku0818
closed
5 months ago
1
如何加载微调之后的checkpoint呢
#228
SmallZhangZhang
closed
5 months ago
1
update orion template
#227
Tendo33
closed
5 months ago
0
internlm2-chat-7b 部署完停不下来,是不是模板的问题啊
#226
Tendo33
closed
5 months ago
4
使用vllm启动server.py时报错
#225
whm233
closed
5 months ago
1
添加InternLM2模型支持
#224
iisquare
closed
5 months ago
1
React Prompt Observation Stop Words判断逻辑优化
#223
lzhfe
closed
5 months ago
1
运行 openai_api.py 启动以后 在执行request_test.py 报错
#222
frog-game
closed
5 months ago
5
运行这个streamlit run streamlit_app.py报错
#221
deauss2017
closed
5 months ago
1
你好 llama-cpp 启动方式支持chatglm3-6b吗
#220
lucheng07082221
closed
5 months ago
1
'indices' to have one of the following scalar types: Long, Int; but got torch.cuda.FloatTensor instead (while checking arguments for embedding)
#219
WallenHan
closed
5 months ago
1
vllm 加载 llama2 设置多卡不生效
#218
Maydaytyh
closed
6 months ago
2
使用 llama-cpp 部署模型推理时报错
#217
Tendo33
closed
5 months ago
4
请问如何支持Qwen1.8B的模型?
#216
lucheng07082221
closed
6 months ago
5
fix process_qwen_messages spell error and unused f_args
#215
Yimi81
closed
6 months ago
0
能解试下这个presence_penalty 参数应该设置多少吗?
#214
lucheng07082221
closed
6 months ago
2
请问它支持langchain中这种方式调用吗:from langchain.llms import OpenAI
#213
whm233
closed
6 months ago
9
💡 [REQUEST] - <title>支持 Mixtral 8x7B
#212
jinghai
closed
5 months ago
0
“添加 TGI 生成接口转发“怎么用?
#211
foxxxx001
closed
5 months ago
1
VLLM已经支持gptq,可以支持gptq吗
#210
Alberoyang
closed
6 months ago
1
baichuan2-13b-chat,正常启动之后,无法使用gpu资源
#209
TangSY
closed
5 months ago
7
Multiple GPU deployment error
#208
Yimi81
closed
5 months ago
0
💡 [REQUEST] - 对于 API 类大语言模型的支持
#207
DevXiaolan
closed
5 months ago
2
"POST /v1/chat/completions HTTP/1.1" 404 Not Found
#206
ocean-zhc
closed
6 months ago
3
能提供模型训练的完整框架吗
#205
lucheng07082221
closed
6 months ago
1
💡 [REQUEST] - 支持tools和tool_choice的适配调用
#204
leoterry-ulrica
closed
5 months ago
1
BaiChuan2Template不支持system prompt,需要添加以下模板支持
#203
DarkDance
closed
6 months ago
1
加载Baichuan2-13B时出现TypeError: unsupported operand type(s) for |=: 'dict' and 'dict'
#202
Zscozer
closed
6 months ago
1
ChatGLM3的输入长度超过8k依然报错
#201
lzhfe
closed
3 months ago
0
使用 ADAPTER_MODEL_PATH 加载 QLoRA 微调的 ChatGLM3 模型失败
#200
Yuanye-F
closed
6 months ago
2
Using llama.cpp engine 能正常启动,但是对话出现 500错误
#199
Fbai700
closed
6 months ago
1
💡 [REQUEST] - 请支持类似FastChat的Controller功能,以便支持多个模型部署
#198
rainsoft
closed
2 months ago
0
普通加载模型方式(非vLLM)推理性能明显比ChatGLM3官方的openai_api.py低
#197
leoterry-ulrica
closed
6 months ago
17
vllm启动qwen72B出错
#196
T0L0ve
closed
6 months ago
2
如何设定本地EMBEDDING_NAME 对应的model path?💡 [REQUEST] - <title>💡 [REQUEST] - <title>
#195
tongcu
closed
6 months ago
1
请问这段代码是否有问题?
#194
TheBobbyliu
closed
6 months ago
1
Previous
Next