search

mediacms-io / mediacms

MediaCMS is a modern, fully featured open source video and media CMS, written in Python/Django and React, featuring a REST API.
https://mediacms.io
GNU Affero General Public License v3.0
2.7k stars 498 forks source link

Option to auto-generate subtitles #553

Closed KyleMaas closed 1 year ago

KyleMaas commented 1 year ago

Describe the feature you'd like Having subtitles easily available without manual intervention would be an incredibly helpful feature for the hearing-impaired as well as visual learners. The way I envision the process for this working is that when you upload a video, you could check a box to optionally take a best guess at subtitles. When checked, upon upload MediaCMS could use something like this to generate a subtitle file and add it to the media:

https://github.com/abhirooptalasila/AutoSub

It's already under a compatible license and seems to use similar dependencies.

Describe alternatives you've considered Most of the other alternatives I'm finding for auto-generating subtitles use Google as a speech recognition backend. This is problematic for several reasons including privacy concerns and the need to generate an API key to use it. By using Mozilla DeepSpeech like AutoSub does above, the subtitles can be generated entirely locally.

KyleMaas commented 1 year ago

Bit more information on this program:

https://towardsdatascience.com/generating-subtitles-automatically-using-mozilla-deepspeech-562c633936a7

mgogoulos commented 1 year ago

I like the idea and I've been thinking of integrating DeepSpeech for a long time. Whisper has been released very recently and is clearly working very well, I've already prepare a small POC using it and soon plan to open a PR

https://github.com/openai/whisper

KyleMaas commented 1 year ago

Very cool. I've been using AutoSub to good effect for a while now, and it's awfully helpful. Even if it requires some manual editing it's still a great way to jumpstart the process.