Utils

[Veldhoen]

• work_it_locally fixes #8. All functionality (Hecate for detection, OpenCV for keyframe extraction, wave&tensorflow for audio extraction and spectogram creation) is in different util modules in this repo. To apply all functionality to a video file, run work_it_locally in a container (as per https://github.com/beeldengeluid/dane-video-segmentation-worker/tree/containerize). This is roughly the code that will be moved to the callback in worker.py. Apart from main, has two extra functions: get_fps and frame_index_to_timecode to transform hecate output (frame indices) into timestamps (ms from start). I use that for audio processing, and it is generally a more generic metric also in a later stage to be able to retrieve matching videos in similarity search.

• hecate_util.py fixes #5. Hecate is called as a subprocess from hecate_util and output is parsed into shot boundaries and keyframe indices. I altered the base_util.py (as copied from https://github.com/beeldengeluid/dane-asr-worker/blob/main/base_util.py) to return stdout - there may be a more elegant solution, I'm open to feedback!

• keyframe_util.py does not correspond to an issue (I presumed Hecate was going to do this part and I never created a separate issue). Based on https://github.com/beeldengeluid/dane-shot-detection-worker/blob/main/hecate.py, I use OpenCV to extract keyframes for all keyframe indices returned by Hecate

• spectogram_utilfixes #3 (and obsoletes #6). Following the example by Teng (https://github.com/beeldengeluid/dane-visual-feature-extraction-worker/blob/main/example.py), apply audio extraction and create spectograms. The major change with respect to Teng's code, is that he generated a spectogram at every 1 second interval from the start of the video, whereas I use a 1 second window around each keyframe. I asserted the functionality is maintained by imposing artificial 'keyframes' at time 500,1500,2500 ms (in unit test, to be committed soon).

I'm still working on some unit tests (for the spectogram utility), will commit them soon

[Veldhoen]

@jblom should I continue to add unit tests to this branch, to enable your reviewing process, or is it cleaner to add them to a separate branch (based on this one)?