xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.
https://inference.readthedocs.io
Apache License 2.0
5.33k stars 431 forks source link

0.15.0视觉模型存在问题,最近会修复吗 #2272

Closed 401557122 closed 1 month ago

401557122 commented 1 month ago

System Info / 系統信息

registry.cn-hangzhou.aliyuncs.com/xprobe_xinference/xinference:v0.15.0

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece?

Version info / 版本信息

0.15.0

The command used to start Xinference / 用以启动 xinference 的命令

docker run -dit -v /data1/llms0.15.0:/root/.xinference -e XINFERENCE_HOME=/root/.xinference -p 9999:9997 --gpus all --shm-size 20g --ipc=host registry.cn-hangzhou.aliyuncs.com/xprobe_xinference/xinference:v0.15.0 xinference-local -H 0.0.0.0 --log-level debug

Reproduction / 复现过程

1.运行qwenvl2模型出现transformer的'Qwen2VLForConditionalGeneration'问题,更新transformer到最新版又出现self._tokenizer = self._processor.tokenizer需要修改为self._tokenizer = self._processor._tokenizer才能部署成功的问题,最后部署成功了,调用又出现参数输入的错误Failed to generate chat completion, detail: [address=0.0.0.0:39506, pid=1510] can only concatenate str (not "list") to str

Expected behavior / 期待表现

请问这是阿里云镜像错误还是这个版本的xinferece有问题

qinxuye commented 1 month ago

@Minamiyama 知道是什么问题吗?

Minamiyama commented 1 month ago

@Minamiyama 知道是什么问题吗?

不知道,官网的demo代码也没见有改

mikck commented 1 month ago

遇到加一

monk-after-90s commented 1 month ago

+1

qinxuye commented 1 month ago

最好是等待 transformers 发版。

401557122 commented 1 month ago

@Minamiyama 能把镜像更新成您的可用的transformer版本吗

Minamiyama commented 1 month ago

@Minamiyama 能把镜像更新成您的可用的transformer版本吗

最好是等待 transformers 发版。

liduang commented 1 month ago

+1,都不敢升级用了

TideDra commented 1 month ago

+1

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 7 days with no activity.

qinxuye commented 1 month ago

Transformers 已经发版。

pip install transformers==4.45.1