search

Lightning-Universe / lightning-transformers

Flexible components pairing 🤗 Transformers with :zap: Pytorch Lightning
https://lightning-transformers.readthedocs.io
Apache License 2.0
607 stars 77 forks source link

Speech recognition tasks #247

Closed rafaelvp-db closed 1 year ago

rafaelvp-db commented 2 years ago

🚀 Feature

Creating a new type of tasks in Lightning Transformers for Speech Recognition (ASR), e.g. Wav2Vec2, etc.

Motivation

Pitch

Currently to train or finetune ASR models using Hugging Face, some extra/boilerplate code is needed (preprocessing, creating vocabularies etc). At the same time, currently ASR is not available in Lightning Transformers. Would make it much easier to train such models using DDP, Horovod etc.

Alternatives

No alternatives AFAIK

Additional context

I've tried to start creating a brand new ASR task (based on the existing tasks for question answering, text classification etc), but struggling a bit with some aspects for instance: how to make this as flexible/reusable as possible, general doubts about Lightning etc. If anyone is willing to review / help / work together: https://github.com/rafaelvp-db/lightning-transformers/tree/feature/asr. I don't have a lot of experience with Lightning so bear with me :)

SeanNaren commented 2 years ago

This is amazing!! It seems you've merged in recent changes to simplify the API, I hope it helps in the process of creating the task.

If you have any questions let me know; if you open the PR I can help to contribute if I get time :)

rafaelvp-db commented 2 years ago

Awesome @SeanNaren! Will do :)

stale[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.