issues
search
xusenlinzy
/
api-for-open-llm
Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, Xverse, SqlCoder, CodeLLaMA, ChatGLM, ChatGLM2, ChatGLM3 etc. 开源大模型的统一后端接口
Apache License 2.0
2.15k
stars
252
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
minicpm启动没问题,推理访问报错
#292
760485464
opened
3 days ago
0
glm-4v启动正常 访问推理报错
#291
760485464
opened
3 days ago
0
使用api-for-open-llm&vllm多卡部署运行Qwen2-7B时报错显存占满
#290
Woiea
opened
5 days ago
5
change the parameter best_of of vllm chat_completion
#289
Tendo33
closed
2 weeks ago
1
glm4 接入dify后无法触发使用工具
#288
he498
opened
2 weeks ago
0
使用 streamer_v2 会造成乱码
#287
Tendo33
opened
2 weeks ago
2
"POST /v1/files HTTP/1.1" 404 Not Found
#286
KEAI404
closed
2 weeks ago
1
使用最新的 vllm 镜像推理qwen2-72B-AWQ 报错
#285
Tendo33
closed
2 weeks ago
4
docker无法下载image
#284
xqinshan
closed
2 weeks ago
1
dcoker部署embedding接口报错:"POST /v1/embeddings HTTP/1.1" 404 Not Found
#283
syusama
closed
3 weeks ago
2
接口请求报错:TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]]
#282
syusama
closed
3 weeks ago
9
glm4用default和vllm方式部署,都不能正常的停止,流式和非流式都有同样的问题,不知道其他朋友是否遇到同样的问题
#281
LiuDQ-wm
closed
3 weeks ago
14
无法运行instruction.py
#280
NCCurry30
opened
3 weeks ago
0
vllm模式推理报错
#279
yeehua-cn
closed
3 weeks ago
2
我现在部署了很多模型,有没有一个webui 界面让我来统一调用部署的模型进行推理
#278
Tendo33
closed
3 weeks ago
3
执行SQL chat时候报ProgrammingError错误
#277
songyao199681
closed
2 weeks ago
1
support vllm0.4.2
#276
FreeRotate
closed
4 weeks ago
0
support vllm==0.4.2
#275
FreeRotate
closed
1 month ago
0
vllm本地部署时,vllm engine启动失败
#274
Ruibn
closed
4 weeks ago
4
什么时候能修复 Qwen 1.5 call function功能了。
#273
skyliwq
opened
1 month ago
0
如何启动接口调用大模型的流式输出 http://127.0.0.1:8080/v1/chat/completions
#272
469981325
closed
3 weeks ago
1
dcoker 部署 vllm 出现 404 Not Found
#271
skyliwq
closed
3 weeks ago
12
EMBEDDING_API_BASE获取不到str expected, not NoneType
#270
chukangkang
closed
2 months ago
5
💡 vllm已经支持流水线并行啦(pipeline parallel),可以极大增加吞吐量,作者可否增加一下vllm的pipeline parallel支持
#269
CaptainLeezz
closed
2 months ago
0
vllm 容器依赖报错
#268
Tendo33
closed
2 months ago
1
Update template.py
#267
claudegpt
closed
2 months ago
0
llama3提问后回答不停止
#266
gptcod
closed
2 months ago
2
找不到图标
#265
lucheng07082221
closed
2 months ago
2
升级后报错
#264
darvsum
closed
2 months ago
1
运行internlm2时报错找不到权重文件,这些文件模型并不提供
#263
760485464
closed
2 months ago
3
关于 api/config.py 中 SETTINGS = Settings()的 bug
#262
Tendo33
closed
2 months ago
1
使用baichuan2-13b-chat模型,回答的乱码,代码写不出来
#261
guiniao
closed
3 weeks ago
2
lifespan not work, cache not cleared
#260
Yimi81
closed
2 months ago
0
ValueError: The model's max seq len (32768) is larger than the maximum number of tokens that can be stored in KV cache (15248). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.
#259
guiniao
closed
2 months ago
1
我想使用的模型不在模型支持列表,是否说明无法使用此项目生成openai的接口
#258
xiaoma444
closed
2 weeks ago
2
MODEL_NAME=qwen2的情况下functions无效
#257
liuyi1213812
closed
2 months ago
3
INFO: 172.20.0.8:60822 - "POST /v1/chat/completions HTTP/1.1" 422 Unprocessable Entity💡 [REQUEST] - <title>
#256
besthong999
closed
2 months ago
1
💡 [REQUEST] - 请求支持CodeLlama-70b-Instruct-hf
#255
Reset816
closed
2 months ago
0
Traceback (most recent call last): File "/Users/yaoxingzhi/Desktop/api-for-open-llm-master/api/server.py", line 1, in <module> from api.config import SETTINGS ModuleNotFoundError: No module named 'api'
#254
779257747
closed
3 months ago
1
💡 [REQUEST] - <title>vllm支持Qwen1.5-14B-Chat-AWQ和Qwen1.5-14B-Chat-GPTQ-int4吗
#253
dickpy
closed
2 months ago
2
用langchain调用bge时报错
#252
Qoooooooooooo
closed
2 months ago
2
[Bug] vLLM 镜像中Pydantic版本冲突
#251
liuyanyi
closed
2 months ago
2
vllm模式启动报错:ImportError: cannot import name 'model_validator' from 'pydantic' (/usr/local/lib/python3.10/dist-packages/pydantic/__init__.cpython-310-x86_64-linux-gnu.so)
#250
syusama
closed
3 months ago
1
docker打包报错:ERROR: failed to solve: process "/bin/sh -c pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple && pip install bitsandbytes --upgrade && pip install vllm==0.3.3 && pip install --no-cache-dir -r /workspace/requirements.txt && pip uninstall transformer-engine -y" did not complete successfully: exit code: 1
#249
syusama
closed
3 months ago
2
💡 [REQUEST] - <请问可以支持一下Xai新发布的Grok-1模型吗>
#248
Hapluckyy
closed
3 weeks ago
1
streamlit 1.32.2 版本新api适配
#247
anonNo2
closed
3 months ago
0
你好,支持mistral模型吗
#246
lucheng07082221
closed
2 months ago
0
Qwen1.5不支持tool_choice
#245
YunmengLiu0
opened
3 months ago
4
InternLM 20B 胡言乱语,什么原因?
#244
jaffe-fly
closed
3 weeks ago
0
TypeError: 'NoneType' object is not subscriptable
#243
deauss2017
closed
2 months ago
0
Next