search

swissspidy / media-experiments

WordPress media experiments
GNU General Public License v2.0
71 stars 1 forks source link

Video captions #227

Open swissspidy opened 8 months ago

swissspidy commented 8 months ago

Ability to create video subtitles on-demand.

Unfortunately it's not possible to do this automatically for new uploads, as the upload queue has no knowledge about which exact block the video is for, and can't update its attributes. And the onChange callback doesn't update the tracks either: https://github.com/WordPress/gutenberg/blob/4494a79e24bfbbcda97ce9af5db3dcb9e81b09f6/packages/block-library/src/video/edit.js#L133-L156

swissspidy commented 8 months ago

And the onChange callback doesn't update the tracks either: WordPress/gutenberg@4494a79/packages/block-library/src/video/edit.js#L133-L156

Maybe GB could, in a backward compatible way, provide the clientID to the upload function somehow.

swissspidy commented 8 months ago

There's also https://github.com/xenova/whisper-web, though it seems more useful for transcripts

swissspidy commented 7 months ago

The current solution seems to work fine, though needs some fine-tuning to ensure proper length (x words/characters per line). Not sure if it does proper sentences to split by, but that could be an option.

swissspidy commented 7 months ago

Would be good to compare the current solution with https://github.com/xenova/whisper-web. The upcoming release there might have better support for word-level timestamps, which would help for subtitles, but even without that it seems to have good output already.

swissspidy commented 2 weeks ago

vosk-browser is quite heavy (5.5MB). There is vosklet, which is a bit smaller (3.5MB)