Could you please offer an example of gyro data for reference?

[Fridaycoder]

Hi, @ElvinC I am trying to apply your program to mobile phones, now I have recorded the video and the corresponding gyro data, but I'm not sure if the format of the gyro data is correct. So could you please offer an example of gyro data and the corresponding video as one demo for checking what the problem is. If so, I would appreciate it a lot. Thank you very much.

[ElvinC]

There's not a generic set in stone format, but there's one used for logger hardware testing purposes, which you can use for testing (note that it might change still). The format is a CSV file containing:

t,gx,gy,gz
7364,15,-28,30
7366,14,-26,27
7368,15,-23,28
7370,17,-23,28
7372,20,-21,25
7374,19,-23,21
7376,20,-21,20
...

Where t is the time in milliseconds, and gx,gy,gz are the gyro rates in a "weird" scaling. To convert from radians per second to this, you do gx = (gx_rad)/(0.070 * np.pi/180) (I was logging raw unscaled sensor data, so we're simulating that scaling). As for the coordinate system, consult this sketch: where green rotations correspond to positive sensor values.

Btw, how did you go about recording the gyro data? And at what sampling rate? I had a look at a few apps, but it seemed like most didn't support high frequency logging.

[Fridaycoder]

Thanks for your reply, actually, my teammate just records the gyro data directly from the phone, not from any apps, the sampling rate is nearly 400 hz, and the raw data is very similar to yours, but is a little different, it is like this:

t, gx, gy, gz 83090679166 -0.113182 -0.128647 -0.191207 83093276250 -0.116847 -0.145751 -0.18632 83095873281 -0.119291 -0.162855 -0.180212

Where t is the time in nanoseconds, and gx, gy, gz is the angular velocity in rads/s(I guess).

Now I have tried tranforming the time to the format from 0, the later time is the difference with the first time, like this:

0 -0.113182 -0.128647 -0.191207 2597084 -0.116847 -0.145751 -0.18632 5194115 -0.119291 -0.162855 -0.180212

Then I read the data using logtype == "gyroflow", and modify the code so as to transform the time to the format in seconds, and just set the gyroscale to 1. Then I run the gyroflow.py and export stabilized videos. However, of the 10 videos I have tried in this way, only 1 video looks like it has been stabilized, but the others look like even more shaky than the raw videos, so I'm wondering where the problem is. I have also tried change the order of gx, gy, gz, but the result is even worse, so the order of gx, gy, gz should be correct. What's more, I have also tried different calibrate files, but the result is similar, so the calibrate doesn't matter much, so I think the problem must be the data format. According to your reply, I guess that maybe I should not transform the time to the format from 0, and just keep it as what it records？But in this way how can gyroflow know the time when the video begins, and how can it deal with that kind of data？

[ElvinC]

You need to either change the parsing to use nanoseconds or change log to milliseconds. If you haven't seen it, then take a look at the relevant parts of the tutorial video: https://youtu.be/NFsTb_f7y8s where the processing stuff is explained in more depth. if you're using a new uncalibrated camera, then you need to make a new lens profile. This is especially the case for phone footage which is quite different from all the existing camera models. Also, you don't need to zero out the file beforehand, gyroflow does that by default already.

[Fridaycoder]

Yeah, actually I have changed the parsing to use nanoseconds, and I have already seen the tutorial on youtube, I think there is no problem in my process steps, but the result is confused. Btw, could you offer one video data and the corresponding gyro data file to me? I just want to check out where the problem is. My email is 864165626@qq.com, thanks a lot.

[ElvinC]

You can try this one from my DSLR (used to see how it handles narrow FOV+rolling shutter) https://we.tl/t-aH2gyN0Cbj Lens preset is Nikon_D5100_Nikkor_35mm_F_1_8_1280x720 expected result is somewhere in the beginning of the tutorial video

[Fridaycoder]

I have tried this one, the result looks very nice, even if I change another error lens preset, the result looks more stable than the raw video, so I think the problem is still the gyro data. I have a few more questions. In this case, did you log the gyro data from your DSLR or an external gyroscope？And did you think the precision of the gyroscope will affect the stabilization results？Since the internal gyroscope in the phone generally has low precision

[ElvinC]

For the DSLR test I used a custom flight controller PCB as a temporary external logger. Most gyroscopes should work for stabilization, but there can be problems if there's too much noise. Another thing to consider is that phone footage typically has a higher amount of rolling shutter distortion, which may be causing issues especially for handheld footage. Feel free to send me a video/gyro/calibration sample if you want me to take a look.

[Fridaycoder]

Here is one of our samples https://we.tl/t-VU5OmlTlwJ. I also attached the result video for the convenience of contrast, the output video looks still very shaky. No idea about what the problem is. Very grateful for your help. Btw, the mobile phone didn't use fisheye lens, so we just use the standard calibrator. What's more, there is no scale in our gyro data, so I just set gyroscale = 1.0.

[ElvinC]

Sorry for not getting back to this sooner. There's now an example of a gyroflow standard format available here: https://github.com/ElvinC/gyroflow/tree/dev/examples