Bad performance when used with DJI Avata 2 videos

[Shadetail]

I've been using gyrfolow ofx plugin with GoPro 11 mini for like a year now. Recently I started using it with my new Avata 2 drone, but whenever I do, I get severe lag, something that I never experienced with GoPro footage, despite GoPro footage being significantly higher resolution at 5.3k vs 4k for Avata 2.

How this manifests, is that when clicking on clip, davinci freezes for a few seconds while it loads the clip, then it works normally unless I try to change a value of parameters in realtime with the mouse, for example if I try to move the horizon roll slider i get 1-2 second lags as the gyroflow is trying to rotate the video accordingly, while with GoPro footage all this has always been smooth and realtime. With GoPro I could have hundreds of gyroflow ofx enabled clips in the timeline, and it would all work smoothly, but with Avata 2 clips, if I click on a clip that I haven't been seeking through in the past few minutes, I get a freeze of up to 7 seconds as I wait for it to load.

I first expected this might be a davinci issue, but it really only happens with gyroflow enabled, and my guess is that it is somehow connected to the lens profile, if not gyro data itself, rather than with the video. As I understand it, this profile has been provided by DJI. Maybe they messed it up somehow?

This issue bad enough that it's making me consider just sticking a GoPro on Avata 2 despite Avata 2 having a larger senzor size, just because it makes editing and previewing the videos so much more difficult.

Here is a link to unedited video file of one of my flights that includes gyro data that can be used for repro: https://drive.google.com/file/d/1Cs7lhTqp2cEgoSkHwKH9l2WLCj8QyySE/view?usp=sharing

[AdrianEddy]

Have you tried the dev build of the plugin? https://gyroflow.xyz/devbuild/

[AdrianEddy]

It was indeed a bug, just updated the nightly build with the fix

[Shadetail]

Hi! Wow that was fast! I downloaded nightly just now of both ofx plugin and standalone gyroflow to test the fix, but found the issue is still happening.

Is it possible that the nightly still didn't finish building the version that contains the fix and I need to wait 24h? I looked around, but couldn't find any way to confirm which version I'm on.

Just in case, here's the video of the bug in action, as I noticed CPU usage graphs behaving very differently with Avata 2 footage vs GoPro footage so I recorded it: https://drive.google.com/file/d/1T2yj8MAZUbVlCAp5xSTPt_VajSjvS9ey/view?usp=sharing

[AdrianEddy]

Hmm, yes that was exactly the bug I fixed. Can you send me gyroflow.log file which is created in C:\Program Files\Common Files\OFX after you close davinci?

[Shadetail]

Here you go: Gyroflow.ofx.log

[AdrianEddy]

Weird, everything looks good in the log. What hardware do you have, is it pc or laptop? Do you have latest GPU drivers?

[Shadetail]

I just updated the nvidia driver from 552.44 to 555.85, but that didn't help. I'm running on a PC in 4k:

• RTX 4090
• Ryzen 9 5950X
• 64 GB RAM (DDR4 (2x32) G.Skill 3600MHz TridentZ NEO, F4-3600C16D-64GTZN)
• 8TB Sabrent Rocket Q NVME SSD
• Windows 11 23H2 (22631.3593)
• Davinci Resolve Studio 18.6.6 build 7

[AdrianEddy]

Try this one and let me know if it's any better https://nightly.link/gyroflow/gyroflow-plugins/workflows/release/main/Gyroflow-OpenFX-windows.zip

[Shadetail]

Thank you for attempting to fix it! ♥ I just tried it and the issue is still present though. Here's a screenshot of how different the cpu usage graphs look when trying to rotate Avata 2 footage vs GoPro11 footage, same as demonstrated in the video before.

[AdrianEddy]

do you have the video files on HDD or SSD?

[AdrianEddy]

Try this updated version now https://nightly.link/gyroflow/gyroflow-plugins/workflows/release/main/Gyroflow-OpenFX-windows.zip

It should now parse each file only once and cache the result in memory

[Shadetail]

I keep all videos on an SSD (Sabrent Rocket Q NVME, plenty fast). I tried the latest version now, and I noticed something that I didn't notice before: video length drastically affects this bug. I'm not sure if one of your updates fixed this bug for short videos only, or if this is just something I haven't noticed before. I recorded a video for you demonstrating this: https://www.youtube.com/watch?v=0UetRT6jpLc

[AdrianEddy]

so looks like this has nothing to do with Avata or GoPro, but is only an issue with clip length. In your video, the GoPro footage is still shorter than the long Avata one

Well, this is expected, when the effect is first applied, Gyroflow has to parse the entire video file and gyro data from it and do all the initial calculations

When changing any parameter, all the smoothing, zooming and distortion calculations have to be done for the entire video, and this is inevitable

it could be possible to limit some calculations to the selected clip instead of entire video, but it's not trvial. I'll look into it

[Shadetail]

GoPro video used as example is 11 min, and that was perfectly usable performance, and the performance was exactly the same as the much shorter 1 min 23 sec GoPro clip.

Here's media info for the longer GoPro clip: File size: 13.9 GiB Duration: 11 min 3 s Overall bit rate: 179 Mb/s Frame rate: 59.940 FPS Width: 5 312 pixels Height: 2 988 pixels Format: HEVC Format profile : Main 10@L6@Main Codec ID: hvc1

So it appears that for GoPro, performance does not scale with video duration, at least not in a way that is noticeable here.

But for Avata2 videos, it went from usable to very laggy when we reached the video length of 5 min 43 sec, which is less than half of the length of the GoPro video that didn't have any issues.

Here is mediainfo for that Avata 2 clip: File size: 5.21 GiB Duration: 5 min 43 s Overall bit rate: 130 Mb/s Frame rate: 59.940 FPS Width: 3 840 pixels Height: 2 880 pixels Format: HEVC Format profile: Main 10@L5@High Codec ID: hvc1

So Avata 2 clip here is significantly shorter and overall lighter, but performs much worse than the much longer heavier GoPro clip.

I noticed now that I had a .gyroflow file next to the gopro file, and removing it made "Load current file" use defaults which did negatively affect performance compared to high performance in the youtube video I shared, but the lag still never exceeded one second, and it's consistent.

While for the 5 min 43 s Avata clip with default settings lag is inconsistent, sometimes the response in instant, but also easily lags up to 6 seconds.

For these two videos, all the settings that are exposed in the plugin are the same, as both are at defaults due to no .gyroflow files present. It makes me suspect that DJI's lens profile might be to blame, as that's definitely different, but I'm not sure if that's something that could even in theory affect performance. And I'm also not sure what else could be different in between them with regards to gyro data or gyroflow settings which might not be exposed in the plugin.

I opened both in gyroflow and compared default settings. I found that "Rolling Shutter Correction" was enabled in for Avata 2 video but disabled for GoPro video:

I tried disabling it for Avata 2 video and saving that as .gyroflow file and loading that in the plugin, but it didn't make the performance any better. I also tried to load GoPro lens profile into Avata 2 video, but that didn't help the performance either.

[AdrianEddy]

ah yes, Rolling shutter correction does add to the calculations quite a bit. Also, the "zoom limit" feature is quite heavy and depending on the type of video, it may be kicking in in the Avata case, but not the GoPro case. You can set the zoom limit to 300 to disable it. You can also try to disable rolling shutter correction and save that in the project file and compare the performance then

[Shadetail]

Rolling shutter correction didn't really make a difference for me, but zoom limit did! Setting it to 300 eliminated the lag entirely, it made even the longest Avata 2 video perform consistently with around 0.25 seconds required for the rotation change.

Too bad that's like the best new feature Gyroflow has.

It's possible that "caching the result in memory" feature you added fixed the large practical aspect of this issue where using many clips in a larger project without any changing of gyroflow settings made jumping around the timeline painfully slow just because all clips have active Gyroflow effect on them. I'll need to set up a project with many clips to test this out since the updated plugin broke backwards compatibility with my previous projects that used gyroflow plugin. (With small number of clips Resolve keeps them all in memory and issue is not apparent). I'll do that soon and let you know what the situation is like.

Slightly offtopic, but speaking of the zoom limit feature, my prefered way of using Gyroflow is to disable dynamic zooming entirely and instead use static zoom that slightly crops into the frame to give Gyroflow something to work with to make the video smoother. It's my understanding that zoom limit feature can still be used here to achieve dynamically changing smoothness value, and some parts of the video still benefit greatly from very large smoothness, while some require very low. "Max smoothness at high velocity" is a binary way of toggling between two smoothness values to somewhat address this, but this new zoom limit feature should allow a full spectrum of smoothness values, in fact it should allow for Gyroflow to select the largest possible smoothness value for every point in time that wouldn't result in going outside of the frame (given that zoom/fov is locked). Am I correct in thinking that this is how this should work in theory? I was playing around with it and was getting somewhat confusing results so far, so I thought I'd check with you if I'm understanding this correctly and just need to play with settings more or if I'm trying to do the impossible for some reason.

[Shadetail]

I'll do that soon and let you know what the situation is like.

I got around to testing this, and unfortunately the change did not significantly improve the situation of moving around the timeline when the timeline consists of short clips (3 second each) from 28 different Avata 2 videos, all of them with gyroflow applied. I thought the only way of currently working like this would be to not use the zoom limit by setting it to 300 on all clips, but even doing that didn't really change much, I still got up to 6 seconds lag spikes when trying to play the timeline and 90% of playback time would be spent frozen. This seems like a hard thing to debug, I'll let you know if I make any further progress on it.

Oh, and btw I just published the video in question, in case you're curious what the use case for this was.