Auto sync not working on RC2, RC3, RC4, (worked perfecly on RC1)

[xan86]

Is there an existing issue for this?

• [X] I have searched the existing issues

Have you tried the latest build?

• [X] I have tried the latest build

Gyroflow version

v1.0.0-rc4

What operating system are you using?

Windows 11 lastest updates

What GPU are you using?

Nvidia Geforce 940MX

What happened?

On versions from RC2 to RC4, even if auto sync aligns gyro and optical flow curves perfectly, the video is not correctly stabilized. Looks like sync preview curves does not match reality. The only way i have found to smooth the video is to disalign sync point. The curves on the timeline doesn't match anymore but stabilization is correct. The same issue happens with every video and relative gyro data in my configuration.

I made a little video capture of this.

https://user-images.githubusercontent.com/50671523/155205428-cb1f28c2-9d51-4b24-98b0-716b13a60348.mp4

All files on this link: Google Drive Folder

With RC1 version worked perfectly with auto sync whithout workarounds.

RC1 Rough gyro offset -7 s RC2-RC4 Rough gyro offset 35 s

I've also exported .gyroflow file from RC1 and from RC4 projects, you can find them in the google drive folder.

My configuration: Camera: CADDX TURTLE - Lens profile added to google drive folder Gyro data: hardware serial Openlog Blackbox (betaflight) Drone: XJB145 - Forward F4 MINI

Video file is CADDX000083.MP4 and gyro file is LOG00085.TXT (different numbers because different recording devices but same flight)

Relevant log output

19:25:41 [INFO] Latest version: v1.0.0-rc4, current version: 1.0.0-rc4
19:25:41 [INFO] Lens profiles directory: "\\\\?\\D:\\Droni\\Gyroflow\\Gyroflow-windows64_rc4\\camera_presets"
19:25:41 [INFO] Loaded 1293 lens profiles in 21.602ms
19:25:42 [INFO] OpenCL Platform: NVIDIA CUDA, Device: NVIDIA Corporation NVIDIA GeForce 940MX
19:25:42 [DEBUG] (1) gyroflow::rendering::ffmpeg_hw: create AV_HWDEVICE_TYPE_CUDA
19:25:42 [DEBUG] (1) gyroflow::rendering::ffmpeg_hw: created ok AV_HWDEVICE_TYPE_CUDA
[src\rendering\mod.rs:47] gpu_type = NVIDIA
19:25:48 [DEBUG] (10) MDKPlayer: [in m_player->onEvent] "reader.buffering" ""
19:25:48 [DEBUG] (11) MDKPlayer: [in m_player->onEvent] "thread.video" ""
19:25:48 [DEBUG] (12) MDKPlayer: [in m_player->onEvent] "thread.audio" ""
19:25:48 [DEBUG] (12) MDKPlayer: [in m_player->onEvent] "decoder.audio" "FFmpeg"
19:25:48 [DEBUG] (12) MDKPlayer: [in m_player->onEvent] "reader.buffering" ""
19:25:48 [DEBUG] (13) MDKPlayer: [in VideoTextureNodePriv::createTexture] QSGRendererInterface::Direct3D11
19:25:48 [DEBUG] (13) MDKPlayer: [in MDKPlayer::sync] created texture QSGPlainTexture(0x167b2d29b60) QSize(1146, 645)
19:25:48 [DEBUG] (11) MDKPlayer: [in m_player->onEvent] "decoder.video" "MFT"
19:25:48 [DEBUG] (13) MDKPlayer: [in m_player->onEvent] "render.video" "1st_frame"
19:25:48 [INFO] surface size: 1280x720
19:25:48 [DEBUG] (13) MDKPlayer: [in VideoTextureNodePriv::createTexture] QSGRendererInterface::Direct3D11
19:25:48 [DEBUG] (13) MDKPlayer: [in MDKPlayer::sync] created texture QSGPlainTexture(0x167b2697570) QSize(1280, 720)
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000016794FB5CC0] st: 0 edit list: 1 Missing key frame while searching for timestamp: 1000
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000016794FB5CC0] st: 0 edit list 1 Cannot find an index entry before timestamp: 1000.
19:26:30 [DEBUG] (2) gyroflow::rendering::ffmpeg_hw: [dec] codec type AV_HWDEVICE_TYPE_DXVA2 0
19:26:30 [DEBUG] (2) gyroflow::rendering: Selected HW backend AV_HWDEVICE_TYPE_DXVA2 (dxva2) with format Some(AV_PIX_FMT_DXVA2_VLD)

[h264 @ 00000167B52CD540] Failed setup for format dxva2_vld: hwaccel initialisation returned error.
19:26:34 [INFO] Setting offset at 54600.0000: 36314.9900 (cost 189560.9242)
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000167B7409380] st: 0 edit list: 1 Missing key frame while searching for timestamp: 1000
[mov,mp4,m4a,3gp,3g2,mj2 @ 00000167B7409380] st: 0 edit list 1 Cannot find an index entry before timestamp: 1000.
19:26:40 [DEBUG] (2) gyroflow::rendering::ffmpeg_hw: [dec] codec type AV_HWDEVICE_TYPE_DXVA2 0
19:26:40 [DEBUG] (2) gyroflow::rendering: Selected HW backend AV_HWDEVICE_TYPE_DXVA2 (dxva2) with format Some(AV_PIX_FMT_DXVA2_VLD)

[h264 @ 00000167B52CB0C0] Failed setup for format dxva2_vld: hwaccel initialisation returned error.
19:26:43 [WARN] Sync point out of acceptable range 1998.989999999998 < 1800
19:26:43 [INFO] Setting offset at 109350.0000: 36086.9900 (cost 264473.1044)

[AdrianEddy]

We now have a robust sync algorithm in the latest version so this can be closed

[xan86]

I'm sorry but i tried the lastest release 1.0.1 and the issue remains exactly the same. Try yourself to stabilize my video in drive folder. The only way to use lastest release's functionalities is to sync with gyroflow RC1, export whith gyro data and import files to 1.0.1 whithout recalculate sync points.

[AdrianEddy]

1.0.1 doesn't include the updated sync algorithm, use dev build https://gyroflow.xyz/devbuild/?autodownload

[AdrianEddy]

What's your camera angle? This parameter is pretty important

[AdrianEddy]

Should be fixed now in gh532

[AdrianEddy]

btw looks like your IMU rotation is: Pitch: 23, Roll: 0, Yaw: -3. That gives me best results in your file

[xan86]

Great job! Autosync is a bit slower and more CPU-consuming but it works fine!

[AdrianEddy]

In gh534 I added a checkbox to disable searching for positive and negative rough offset - this speeds up the sync if you already know that the offset is positive