lay295 / TwitchDownloader

Twitch VOD/Clip Downloader - Chat Download/Render/Replay
MIT License
2.68k stars 261 forks source link

Any way to speed up chat rendering? #949

Closed Shrezno closed 8 months ago

Shrezno commented 8 months ago

Checklist

Write stuff here

I have a very beefy pc and can render 2 hour videos in various video editing softwares in 20 minutes but with TwitchDownloader it takes like 1h50m to render a 2 hour chat file. Feel like something is bottlenecking and wanted to ask if there is anything that can be done to speed it up. I am rendering at pretty high resolution at 60fps with a 0.1 update rate and a mask generated so it might be fair that its somewhere in the 1.1x speed range

ScrubN commented 8 months ago

Feel like something is bottlenecking

Yeah, unfortunately the renderer is about as fast as it can be without a full rewrite or a hail-mary optimization technique. Because we do not want to compile FFmpeg binaries for each platform, it means we must spawn an FFmpeg process and write to it over a pipe, which means the memory for each frame needs to be copied rather than just passing a pointer to the encoder.

In addition to this, generate mask more than doubles the time it takes to render because it needs to write to a separate FFmpeg process, however the mask frame buffer also overwrites the original frame buffer, so that disables a different optimization too.

In regards to high resolution, the time it takes FFmpeg to encode is obviously part of the equation, however the renderer is also entirely on the CPU. I would like to add GPU rendering support, but the current architecture isn't optimized for it and if I were to ignore anyways that then the performance gain would probably be heavily impacted by an absurd amount of non-batched draw calls.

In regards to update rate, if you do not have dispersion enabled then update rate will be treated as if it is set to 1.0. If you do have dispersion enabled, it might increase render times by ~1-5%? Dispersion no longer has a huge impact because of an optimization I implemented sometime last year, however it still is not free.

Framerate is not as easy to quantify because it adds more work for FFmpeg, more frame copying, more frames for animated emotes, and more opportunities for the comment list to update.

If you have image sharpening enabled then that will add around 10% to the render time.

Lastly, the encoding time of the chosen codec is also important to consider. In my experience, H264 tends to be the fastest.

I really wish the renderer was faster too. I'll add GPU rendering to my backlog, though I can't promise when it will be released.

Shrezno commented 8 months ago

Thanks for the feedback, really useful software

ScrubN commented 8 months ago

if you do not have dispersion enabled then update rate will be treated as if it is set to 1.0

Sorry, minor correction. This is only true for any chats downloaded from Twitch after ~November 2022

Thanks for the feedback, really useful software

I'm glad you get good use out of it. I've noticed some big streamer editors stopped using the chat renderer recently and it makes me a bit sad.