gyroflow / gyroflow

Video stabilization using gyroscope data
https://gyroflow.xyz
GNU General Public License v3.0
6.22k stars 264 forks source link

10+ Minute Long Videos take Forever To Load + Takes all the ram in DaVinci Resolve #813

Open tlhd420 opened 2 months ago

tlhd420 commented 2 months ago

Is there an existing feature request for this?

Description

Hello all

First let me start of with saying that I am grateful for the team behind Gyroflow as it's been able to produce phenomenal footage.

I typically use it for real estate & automotive content. When I'm doing real estate I do have a talking head and it's a full walk-through of the house so the video tends to be 15-25 minutes long. I've noticed a lot but never thought much about it, how slow it is to loading the A-roll clip vs the short B-roll clips

And whenever I am playing back footage in DR and it goes from B to A-roll, DR will usually pause and the ram usage will spike and there will be a delay of around 10-15 seconds before playback resumes. Sometimes longer, which can be a nuisance whenever I'm finalizing the video with the client. This is especially apparent whenever going from the edit page or color page as it'll just take even longer Even when exporting it does the same pauses between B-Roll and A-roll, albeit not nearly as long

I only use the plugin on DR + Gyroflow 1.5.4 from the Windows Store. Running Windows 11, Ryzen 9 7900X + 64GB Ram + RTX 3090 and have had my ram completely maxed out at 64GB and it crashes the export/DR

Screenshot (125)

Screenshot 2024-02-13 135316

My hopefully, temporary fix is just cutting the video when I know I'm going to cover the gap, but I'd prefer to just have the one video clip as it would be less work

Please keep up the great work & thank you for reading Whether or not you guys can actually implement or improve this won't change the fact I will continue to support and use Gyroflow

Good day!

AdrianEddy commented 2 months ago

What camera do you use? Gyro data parsers are different for every camera

tlhd420 commented 1 month ago

What camera do you use? Gyro data parsers are different for every camera

My apologies, I don't know how I didn't manage to add that to the post

I'm using a Sony FX30 with no IBIS & if it matters, XAVC-HS (h.265) 10-Bit 4:2:0

AdrianEddy commented 1 month ago

well, there's definitely room for improvement here, because this is also an issue on iOS and Android where RAM is very limited

However, it's not that easy, we have to make compromise somewhere with computation time vs memory. The gyro data needs to be held in memory, and with 20 min video, that's a lot of data (Sony samples the gyro at 2000 Hz, that's 2000 ~30 byte samples per second), not to mention that that gyro data has to be integrated into quaternions (first computation), then it needs to be computed two times forward and backward for smoothing algorithm, and then it needs to go through it again to calculate optimal zooming amount, and on top of that, it needs to hold the frame transform values for every row of the frame, for rolling shutter correction. That's a lot of data and the worst part is that it's inevitable

We can store less in memory, but we'd have to do the re-compute every time it's needed (that's the slowdown on cuts you're seeing), or we can store everything in memory but then that takes a lot of RAM.

I'll keep this issue open because it's definitely relevant, but I'm not sure how much of an improvement I can make here. One easy solution would be to add a slider in the plugin to balance between compute time vs RAM and then every user would be able to set the value for his needs. I don't know, I'll think about it

tlhd420 commented 1 month ago

well, there's definitely room for improvement here, because this is also an issue on iOS and Android where RAM is very limited

However, it's not that easy, we have to make compromise somewhere with computation time vs memory. The gyro data needs to be held in memory, and with 20 min video, that's a lot of data (Sony samples the gyro at 2000 Hz, that's 2000 ~30 byte samples per second), not to mention that that gyro data has to be integrated into quaternions (first computation), then it needs to be computed two times forward and backward for smoothing algorithm, and then it needs to go through it again to calculate optimal zooming amount, and on top of that, it needs to hold the frame transform values for every row of the frame, for rolling shutter correction. That's a lot of data and the worst part is that it's inevitable

We can store less in memory, but we'd have to do the re-compute every time it's needed (that's the slowdown on cuts you're seeing), or we can store everything in memory but then that takes a lot of RAM.

I'll keep this issue open because it's definitely relevant, but I'm not sure how much of an improvement I can make here. One easy solution would be to add a slider in the plugin to balance between compute time vs RAM and then every user would be able to set the value for his needs. I don't know, I'll think about it

And I am grateful for your reply! If you guys are unable to improve it like I said previously, not a deal breaker! I'll just work around those limitations by doing 5-10miniute clips (and so far, that is working much better)

Thank you for letting me know how it all works, I wasn't sure how much was happening behind the screen but jebus, y'all have got your hands full. Don't work too hard ;)

Much appreciated, have a great day!