search

ohmeow / blurr

A library that integrates huggingface transformers with the world of fastai, giving fastai devs everything they need to train, evaluate, and deploy transformer specific models.
https://ohmeow.github.io/blurr
Apache License 2.0
289 stars 34 forks source link

Would it be possible to use this library to train a Hugging Face Wav2Vec2 model using FastAI? #28

Closed Y-T-G closed 3 years ago

Y-T-G commented 3 years ago

I see that this is primarily intended to be used for transformers that deal with textual data.

I was wondering whether it can be still used to train a Wav2Vec2 hugging-face transformer model using FastAI.

I was able to modify the get_hf_objects function enabling it to retrieve the objects related to Wav2Vec2 transformer.

How much more modification do you think would be necessary to get it to basically work on audio data?

ohmeow commented 3 years ago

I think the only necessary changes would come in the data blocks ... basically the preparation of raw data to tensors for training (the datablock code in place is strictly for text).

If you get things working and want to make a PR, that would be awesome. If you look at the code, you can see things are divided between data modules and training/modeling modules. Basically you'd have to build a data module for making the audio data ready for training, and then a modeling module for anything specific for training (e.g., show_results, etc..). Lmk.

Thanks - wg

On Mon, Feb 22, 2021 at 11:13 PM Y-T-G notifications@github.com wrote:

I see that this is primarily intended to be used for transformers that deal with textual data.

I was wondering whether it can be still used to train a Wav2Vec2 hugging-face transformer model using FastAI.

I was able to modify the get_hf_objects function enabling it to retrieve the objects related to Wav2Vec2 transformer.

How much more modification do you think would be necessary to get it to basically work on audio data?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ohmeow/blurr/issues/28, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAADNMHSELODSPQ3U7HFYE3TANIQ3ANCNFSM4YB6CIFQ .

Y-T-G commented 3 years ago

I see. I think that would be difficult for me given my limited knowledge. But I might try and see if I somehow am able to get it working.

ohmeow commented 3 years ago

Closing this out for now.