ThioJoe / Auto-Synced-Translated-Dubs

Automatically translates the text of a video based on a subtitle file, and then uses AI voice services to create a new dubbed & translated audio track where the speech is synced using the subtitle's timings.

GNU General Public License v3.0

1.56k stars 156 forks source link

whisperX to create transcripts #28

Open JvSdv opened 1 year ago

JvSdv commented 1 year ago

PLEASE ONLY SUBMIT PULL REQUESTS FOR BUG FIXES OR LEGITIMATELY USEFUL ADDITIONAL FUNCTIONALITY

I know many of you are enthusiastic to help but I don't want this project to become bloated

Type of change

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

Proposed Changes

Add transcription of videos in english using whisperx an upgrade of the original whisper, with well defined timestamsps

Additional Info

Check out how to use whisperx with other languages and how to use it at (https://github.com/m-bain/whisperX)

Checklist:

[x] I have commented my code, particularly in hard-to-understand areas
[x] I have tested that the code works

Required:

By contributing to this project, you agree to the terms of the GPLv3 license, and agree to grant the project owner the right to also provide or sell this software, including your contribution, to anyone under any other license, with no compensation to you.

ThioJoe commented 1 year ago

Instead of adding this to requirements.txt, I'll probably want to implement it optionally somehow. Maybe add a message to be displayed when the script is run saying how to install it. Mostly because it would require downloading big files like for the model, and also not everyone would have a GPU that would even work with it.