search

collabora / WhisperSpeech

An Open Source text-to-speech system built by inverting Whisper.
https://collabora.github.io/WhisperSpeech/
MIT License
3.98k stars 216 forks source link

Update model metadata and information on the Hugging Face Hub. #42

Open Vaibhavs10 opened 10 months ago

Vaibhavs10 commented 10 months ago

Hi @jpc and team,

Congratulations on such a brilliant model, and thanks for going the extra mile to make sure it is easily accessible. I'm VB I lead the advocacy efforts for Audio at Hugging Face. Adding a model card to the checkpoint on the Hub would be great, and adding some metadata allows users to search for the model. An example model card would be suno/bark.

Note: the model card's content could be similar to the GitHub README.

We've seen this as a great way to increase the model's visibility, too.

In addition to that, it'd be great to create a demo for it on the Hugging Face spaces. We'd be happy to support you with GPU grants (for inference on the space) to help further democratise this model's use.

I'm just a ping away if you need any help.

Cheers! VB

jpc commented 10 months ago

Hi, thanks a lot for the invite! :) A Huggingface spaces demo would probably be more accessible than Google Collab. Having a more performant GPU available for people to play with would also help.

I do have a question about Spaces. We recently added support for torch.compile which makes the inference quite a bit more efficient. Unfortunately it requires a warmup phase that takes 20-30 seconds on first inference (and the compiled model cannot be serialized yet). Afterwards each additional inference is super fast (12x faster than realtime on a 4090). Do you know how this would work with Spaces? As far as I understand a space is only spun up as needed?

Vaibhavs10 commented 9 months ago

Hey hey! @jpc - Sorry for the delay in responfding to you. Yes, The HF space should be able to handle the compilation step. Since the model compilation would happen before the space is ready to use. So, there will be no degradation in UX overall.

Happy to help with it if you want.

If you're game then we can also have a shared slack channel to discuss about further potential collaborations and how Hugging Face can help scale your and teams experiments better.

I believe my colleague Yoach would've reached out to you, if not then feel free to send me your Email: Vaibhav[at]huggingface[dot]co and I'll set up a channel to discuss further.