TMElyralab / MuseTalk

MuseTalk: Real-Time High Quality Lip Synchorization with Latent Space Inpainting
Other
2.5k stars 305 forks source link

Support for other whisper models specifically Whisper-medium #170

Open devs-hooked opened 1 month ago

devs-hooked commented 1 month ago

The problem with using whisper models other than tiny is that the pre-trained models/weights do not work properly with other versions. The encoder and checkpoint functions cause problems due to mismatch in parameter sizes. Are there any different pre-trained weights/models that can work with other whisper models? I've tried searching for another version of dwpose , etc models but no result.

image

czk32611 commented 1 week ago

Could I ask why would you want to use other whisper models?