ybsbbw
commented
2 months ago 我这边用webUI启动emedding模型后，0.11.0版本调用时是一样的问题，换到最新的0.14.3后，报 Traceback (most recent call last): File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xinference/api/restful_api.py", line 1189, in create_embedding embedding = await model.create_embedding(body.input, kwargs) File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xoscar/backends/context.py", line 227, in send return self._process_result_message(result) File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xoscar/backends/context.py", line 102, in _process_result_message raise message.as_instanceof_cause() File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xoscar/backends/pool.py", line 659, in send result = await self._run_coro(message.message_id, coro) File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xoscar/backends/pool.py", line 370, in _run_coro return await coro File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xoscar/api.py", line 384, in on_receive return await super().on_receive(message) # type: ignore File "xoscar/core.pyx", line 558, in on_receive__ raise ex File "xoscar/core.pyx", line 520, in xoscar.core._BaseActor.on_receive async with self._lock: File "xoscar/core.pyx", line 521, in xoscar.core._BaseActor.on_receive with debug_async_timeout('actor_lock_timeout', File "xoscar/core.pyx", line 526, in xoscar.core._BaseActor.__on_receive result = await result File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xinference/core/utils.py", line 45, in wrapped ret = await func(*args, *kwargs) File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xinference/core/model.py", line 90, in wrapped_func ret = await fn(self, args, kwargs) File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xinference/core/model.py", line 572, in create_embedding return await self._call_wrapper_json( File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xinference/core/model.py", line 401, in _call_wrapper_json return await self._call_wrapper("json", fn, *args, kwargs) File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xinference/core/model.py", line 114, in _async_wrapper return await fn(*args, *kwargs) File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xinference/core/model.py", line 418, in _call_wrapper ret = await asyncio.to_thread(fn, args, kwargs) File "/home/wpz/anaconda3/envs/llm/lib/python3.9/asyncio/threads.py", line 25, in to_thread return await loop.run_in_executor(None, func_call) File "/home/wpz/anaconda3/envs/llm/lib/python3.9/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xinference/model/embedding/core.py", line 378, in create_embedding all_embeddings, all_token_nums = encode( File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/xinference/model/embedding/core.py", line 260, in encode elif model.default_prompt_name is not None: File "/home/wpz/anaconda3/envs/llm/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1695, in getattr raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'") AttributeError: [address=0.0.0.0:36191, pid=972983] 'SentenceTransformer' object has no attribute 'default_prompt_name'
System Info / 系統信息
cuda12.2 运行llm模型正常
Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？
Version info / 版本信息
Release: v0.13.2 Commit: 880929cbbc73e5206ca069591b03d9d16dd858bf
The command used to start Xinference / 用以启动 xinference 的命令
xinference launch --model-name bge-m3 --model-type embedding error~ Model not found in the model list, uid: glm4-1-0
运行llm模型正常
Reproduction / 复现过程
1.xinference launch --model-name bge-m3 --model-type embedding error~ Model not found in the model list, uid: glm4-1-0
Expected behavior / 期待表现
launch embedding