search

yeyupiaoling / Whisper-Finetune

Fine-tune the Whisper speech recognition model to support training without timestamp data, training with timestamp data, and training without speech data. Accelerate inference and support Web deployment, Windows desktop deployment, and Android deployment
Apache License 2.0
803 stars 126 forks source link

whisper-medium微调之后，转换模型失败 #84

Closed chengminglei77 closed 3 weeks ago

chengminglei77 commented 1 month ago

我在微调之后，并且执行了python merge_lora,最终生成了model.safesensors模型，在通过如下代码转化pt格式时出现了下述情况。 def save_model(model, download_path=whisper_download_path):

创建whisper模型实例

model = whisper.load_model(model, download_root=download_path)
state_dict = load_safetensors(safetensors_path)
# 加载模型状态字典
model.load_state_dict(state_dict)

# 将模型保存为pt格式
pt_path = os.path.join(whisper_download_path, 's_medium.pt')
torch.save(model, pt_path)

if name == 'main': save_model('medium')

问题： Missing key(s) in state_dict: "encoder.positional_embedding", "encoder.conv1.weight", "encoder.conv1.bias", ... Unexpected key(s) in state_dict: "model.decoder.embed_positions.weight",

yeyupiaoling commented 1 month ago

这不能转换的吧？你哪里找的转换代码？