search

antiboredom / videogrep

automatic video supercuts with python
https://antiboredom.github.io/videogrep
Other
3.33k stars 257 forks source link

Render search fragments as WebVTT file #121

Closed bencomp closed 1 year ago

bencomp commented 1 year ago

It's not SRT, but otherwise this fixes #23.

Based on the algorithm for creating the correct timestamps in XML output, this uses the results from videogrep.videogrep.search to create a .vtt file using the same basename as the output filename.

To enable this export, users need to set the --export-vtt or -ev flag. This is evaluated after creating the supercut, so it won't work when other flags/options are used to do anything other than create a supercut.

antiboredom commented 1 year ago

thank you for this! I'm happy to add this in, but could you add a few tests beforehand? All the tests are located in a single file here: https://github.com/antiboredom/videogrep/blob/master/tests/test_videogrep.py

(if you have any questions about this please just let me know)

bencomp commented 1 year ago

I added a test that makes some basic assertions about the resulting WebVTT file (based on the test_videogrep test). I'm afraid my computer architecture and lack of manifesto.mp4 prevents me from running the test successfully. The tests did run with numpy 1.23.5, pytest 7.2.0 and imageio 2.22.4, but then fail because of the missing video. Are these checks you would deem appropriate and could you run them?

antiboredom commented 1 year ago

oops! I should probably include that in the repo!!! In the meantime, here's the file...

https://user-images.githubusercontent.com/344861/205760866-cc64ddc4-1f53-4f3f-9e65-30ffc8e65fd0.mp4

I'm getting a few errors - let me know if you're able to run the tests with that file in place, otherwise I will take a crack at it!

bencomp commented 1 year ago

Thanks, I can now run the tests. (test_cli is also failing for me, so maybe upgrading some dependencies didn't work out in that respect.)

I can fix my earlier assertions. The number of lines in the resulting .vtt file depends on the way I read the file and split the lines. The end timestamp of the last cue is off by .02 seconds. In longer supercuts I have seen that the subtitles drift as well, even though padding has been performed. I guess the issue may be in ffmpeg cutting by frames instead of exact subseconds, causing accumulation of rounding errors. That's a fix for after work hours.

bencomp commented 1 year ago

I took a few hours to try to make the new test pass by 'snapping' the cues to frame times, but it is more difficult than it looked. The best result is either 6.2 seconds (0.04 seconds too low) or 6.28 seconds (0.04 seconds too high), i.e. off by 1 frame length in the 25 fps manifest video. I haven't tested on longer videos yet, because the ones I have are 30 fps.

So currently my logic is flawed, or the test should allow being off by 1 frame length. Anyway, for these calculations I need the framerate to get the timing correct for all videos. That requires more work, like loading the video again to get the framerate, or adding fps as additional key in the segment dictionaries that I give to vtt.render(segments, outputfile) (the results of videogrep.search). What do you think, @antiboredom ?

antiboredom commented 1 year ago

hi - sorry for the rather extreme delay, but I've made a few quick fixes and merged. Thanks for your contribution!!!