xusenlinzy api-for-open-llm issues

xusenlinzy / api-for-open-llm

Openai style api for open large language models, using LLMs just as chatgpt! Support for LLaMA, LLaMA-2, BLOOM, Falcon, Baichuan, Qwen, Xverse, SqlCoder, CodeLLaMA, ChatGLM, ChatGLM2, ChatGLM3 etc. 开源大模型的统一后端接口

Apache License 2.0

2.36k stars 270 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

找不到图标

#265 lucheng07082221 closed 7 months ago
2
升级后报错

#264 darvsum closed 7 months ago
1
运行internlm2时报错找不到权重文件，这些文件模型并不提供

#263 760485464 closed 7 months ago
3
关于 api/config.py 中 SETTINGS = Settings()的 bug

#262 Tendo33 closed 7 months ago
1
使用baichuan2-13b-chat模型，回答的乱码，代码写不出来

#261 guiniao closed 5 months ago
2
lifespan not work, cache not cleared

#260 Yimi81 closed 7 months ago
0
ValueError: The model's max seq len (32768) is larger than the maximum number of tokens that can be stored in KV cache (15248). Try increasing `gpu_memory_utilization` or decreasing `max_model_len` when initializing the engine.

#259 guiniao closed 7 months ago
1
我想使用的模型不在模型支持列表，是否说明无法使用此项目生成openai的接口

#258 xiaoma444 closed 5 months ago
2
MODEL_NAME=qwen2的情况下functions无效

#257 liuyi1213812 closed 7 months ago
3
INFO: 172.20.0.8:60822 - "POST /v1/chat/completions HTTP/1.1" 422 Unprocessable Entity💡 [REQUEST] - <title>

#256 besthong999 closed 7 months ago
1
💡 [REQUEST] - 请求支持CodeLlama-70b-Instruct-hf

#255 Reset816 closed 7 months ago
0
Traceback (most recent call last): File "/Users/yaoxingzhi/Desktop/api-for-open-llm-master/api/server.py", line 1, in <module> from api.config import SETTINGS ModuleNotFoundError: No module named 'api'

#254 779257747 closed 8 months ago
1
💡 [REQUEST] - <title>vllm支持Qwen1.5-14B-Chat-AWQ和Qwen1.5-14B-Chat-GPTQ-int4吗

#253 dickpy closed 7 months ago
2
用langchain调用bge时报错

#252 Qoooooooooooo closed 7 months ago
2
[Bug] vLLM 镜像中Pydantic版本冲突

#251 liuyanyi closed 7 months ago
2
vllm模式启动报错：ImportError: cannot import name 'model_validator' from 'pydantic' (/usr/local/lib/python3.10/dist-packages/pydantic/__init__.cpython-310-x86_64-linux-gnu.so)

#250 syusama closed 8 months ago
1
docker打包报错：ERROR: failed to solve: process "/bin/sh -c pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple && pip install bitsandbytes --upgrade && pip install vllm==0.3.3 && pip install --no-cache-dir -r /workspace/requirements.txt && pip uninstall transformer-engine -y" did not complete successfully: exit code: 1

#249 syusama closed 8 months ago
2
💡 [REQUEST] - <请问可以支持一下Xai新发布的Grok-1模型吗>

#248 Hapluckyy closed 5 months ago
1
streamlit 1.32.2 版本新api适配

#247 anonNo2 closed 8 months ago
0
你好，支持mistral模型吗

#246 lucheng07082221 closed 7 months ago
0
Qwen1.5不支持tool_choice

#245 YunmengLiu0 opened 8 months ago
4
InternLM 20B 胡言乱语，什么原因?

#244 jaffe-fly closed 5 months ago
0
TypeError: 'NoneType' object is not subscriptable

#243 deauss2017 closed 7 months ago
0
docker vllm运行qwen1.5-7b-chat无法部署，报错：Fatal Python error: Bus error

#242 syusama opened 9 months ago
2
Qwen1.5推理报错RuntimeError: cannot reshape tensor of 0 elements into shape [-1, 0] because the unspecified dimension size -1 can be any value and is ambiguous

#241 syusama opened 9 months ago
6
Qwen1.5 提示模板缺少默认 system message

#240 liuyanyi closed 9 months ago
1
34b 模型，用int4 ，vllm进行推理，单张24G 4090GPU，显示显存不足

#239 haohuisss closed 7 months ago
10
Qwen2运行报错

#238 yinggoga closed 9 months ago
1
fix for max_tokens larger than max_model_len using vllm engine

#237 lzhfe closed 9 months ago
0
使用 vllm 推理，在文本末尾缺失信息

#236 parasol-ry closed 7 months ago
10
Qwen1.5-7B-Chat 使用API呼叫 completions 功能未能生成下文

#235 kanslor closed 7 months ago
2
Update the version of transformer in requirements.txt

#234 Tendo33 closed 9 months ago
0
💡 [REQUEST] - <能否实现一个功能，就是api一段时间无调用的时候，自动卸载到cpu甚至硬盘>

#233 Tongjilibo closed 5 months ago
0
Fix vllm stream function call

#232 lzhfe closed 9 months ago
0
vllm模式function call流式返回异常

#231 lzhfe closed 9 months ago
1
Internlm2-chat-7b无法生成正确的回复

#230 AlanLu0808 closed 7 months ago
2
internlm2-chat-20b，成功运行后，收到请求无回复

#229 deku0818 closed 10 months ago
1
如何加载微调之后的checkpoint呢

#228 SmallZhangZhang closed 10 months ago
1
update orion template

#227 Tendo33 closed 10 months ago
0
internlm2-chat-7b 部署完停不下来，是不是模板的问题啊

#226 Tendo33 closed 10 months ago
4
使用vllm启动server.py时报错

#225 whm233 closed 10 months ago
1
添加InternLM2模型支持

#224 iisquare closed 10 months ago
1
React Prompt Observation Stop Words判断逻辑优化

#223 lzhfe closed 10 months ago
1
运行 openai_api.py 启动以后在执行request_test.py 报错

#222 frog-game closed 10 months ago
5
运行这个streamlit run streamlit_app.py报错

#221 deauss2017 closed 10 months ago
1
你好 llama-cpp 启动方式支持chatglm3-6b吗

#220 lucheng07082221 closed 10 months ago
1
'indices' to have one of the following scalar types: Long, Int; but got torch.cuda.FloatTensor instead (while checking arguments for embedding)

#219 WallenHan closed 10 months ago
1
vllm 加载 llama2 设置多卡不生效

#218 Maydaytyh closed 11 months ago
2
使用 llama-cpp 部署模型推理时报错

#217 Tendo33 closed 10 months ago
4
请问如何支持Qwen1.8B的模型？

#216 lucheng07082221 closed 11 months ago
5

Previous Next