Belle-whisper-large-v3-zh 运行失败，Error no file named model.safetensors found in directory

xorbitsai / inference

Replace OpenAI GPT with another LLM in your app by changing a single line of code. Xinference gives you the freedom to use any LLM you need. With Xinference, you're empowered to run inference with any open-source language models, speech recognition models, and multimodal models, whether in the cloud, on-premises, or even on your laptop.

https://inference.readthedocs.io

Apache License 2.0

4.77k stars 375 forks source link

Belle-whisper-large-v3-zh 运行失败，Error no file named model.safetensors found in directory #2094

Closed reinforceli closed 1 day ago

reinforceli commented 4 weeks ago

System Info / 系統信息

服务端：python310, torch2.3.1+cu121, windows10 客户端使用python38调用 xinference-client

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

[ ] docker / docker
[X] pip install / 通过 pip install 安装
[ ] installation from source / 从源码安装

Version info / 版本信息

xinference-client-0.14.1.post1 xinference 0.14.1.post1

The command used to start Xinference / 用以启动 xinference 的命令

单节点启动 xinference-local

Reproduction / 复现过程

Expected behavior / 期待表现

能正常运行

reinforceli commented 4 weeks ago

换成 transformers-4.40.0 可以运行，transformers-4.43 版本就报错

github-actions[bot] commented 3 weeks ago

This issue is stale because it has been open for 7 days with no activity.

github-actions[bot] commented 2 weeks ago

This issue was closed because it has been inactive for 5 days since being marked as stale.

Halflifefa commented 2 weeks ago

换成 transformers-4.40.0 可以运行，transformers-4.43 版本就报错

原因是xinference/model/audio/whisper.py中设置了使用safetensors，但模型权重是bin，不是safetensors，需要改一下该文件的60行代码，这应该是一个bug

# 原始代码
use_safetensors=True,
# 改为False
use_safetensors=False,

skytodmoon commented 3 days ago

mark.met the same problem; 1.Change the code 2.Change the pytorch.bin