Split and Stitch

[gmarchand]

Video encoding can be a computationally intensive process, especially for large ultra high definition (UHD) video files. To speed up encoding time, the video file can be divided into segments that are encoded in parallel across multiple compute nodes.

An orchestration function is responsible for splitting the video file into segments and distributing those segments to available encoder nodes. The function factors in the video resolution, bitrate, number of available compute nodes, and other parameters to determine the optimal number of parallel encoder nodes to utilize.

Each encoder node is assigned one or more video segments to encode independently. This allows different sections of the video to be processed concurrently, reducing overall encoding time compared to encoding the full video sequentially on a single machine.

Once the nodes complete their assigned encoding tasks, the orchestration function stitches the individually encoded video segments back together into the final output video file. Additional processing such as error checking across segments may also be performed to ensure consistency and quality across the full video.

Efficient parallelization relies on dividing workload intelligently while minimizing inter-node coordination. For video encoding, this involves parsing the file into meaningfully sized segments that nodes can work on without extensive communication or shared state. The orchestrator handles coordination and state management between the encoder pool, optimizing overall encoding throughput.

References :

• https://ccremer.github.io/clustercode/clustercode/index.html
• https://community.gumlet.com/t/how-to-split-a-video-into-chunks-based-on-duration-using-ffmpeg/387/7
• https://copyprogramming.com/howto/use-ffmpeg-to-split-video-clip-code-example
• https://github.com/bfansports/CloudTranscode
• https://github.com/HaveAGitGat/Tdarr
• https://patademahesh.github.io/Distributed-Video-Transcoding/
• https://blog.dustinkirkland.com/2014/07/scalable-parallel-video-transcoding-on.html
• https://medium.com/google-cloud/event-driven-ffmpeg-transcoding-a-modern-solution-with-gcp-42995d5c3dbb

[gmarchand]

https://xilinx.github.io/video-sdk/v1.5/examples/ffmpeg/tutorials.html#faster-than-real-time

When processing file-based video clips, it is possible to run faster than real time (FTRT) by using a map-reduce approach. With this method, the file-based video clip is split into multiple smaller segments, and each of these segments is individually transcoded. The more devices are available, the more segments can be processed in parallel and the faster the process is. While there is some overhead in “splitting” the clip into segments, and “stitching” the results of each segment into a single output file, these costs are almost always outweighed by the improvement in FPS.