drovp / upscale

Drovp plugin for upscaling images with Real-ESRGAN or waifu2x neural networks.
1 stars 0 forks source link

Upscale video #1

Closed tomasklaen closed 2 years ago

tomasklaen commented 2 years ago

Upscaling videos is theoretically doable, but practically very laborious to implement. Dunno when I'll have enough resolve to take a stab at it.

I need to figure out how to split video into frames, but in a streaming way so that they arrive one by one, waiting for previous to get upscaled and encoded into output video before next one is extracted. Extracting all the frames and only then upscaling is a no-go, that'd murder harddrives.

Then ensure the output gets all the assets (audio tracks, subtitle tracks, covers, ...) from the original, while handling all the necessary conversions:

Ugh.

tomasklaen commented 2 years ago

This is now implemented in a non-efficient way: extracting all of the frames, upscaling them, and then re-combining into a new video.

As far as I know, the tooling doesn't allow to do this efficiently, one by one.

Specifically when I attempt to pipe images into ffmpeg through stdin, it stalls if they don't arrive fast enough, which in this case would be always. Also, from discussion on #ffmpeg irc, piping images to stdin might simply break because of stdin buffer size constraints, so this is just not doable for multiple reasons, and all images have to be ready before I try encoding them into a video...

Oh well, hope anyone trying to upscale long videos has enough hard drive space 😢.