Raspberry PI 4B + OpenVINS: Different results with same parameters

[ClementLASSENA]

Hi,

First of all thank you very much for this impresive contribution which helps the whole community. If you allow it I need your help. I'll try to be clear as possible in my issue : Since one week I am trying to settle well my setup but i am having problems.

My setup : open_vins + Mono rolling shutter camera (25 fps) + Consummer grade IMU (100Hz) + Raspberry PI 4

Is my installation good ? : I think, indeed the euroV202 dataset gives me a pretty good trajectory even in mono.

My isues :

1. After a lot of modifications of parameters, my trajectory began to converge. But, if I launch twice the same bag with same parameters I often obtain two different trajectories. Is it normal ? Sometimes the trajectory is very bad ( almost divergent )

Sometimes the trajectory almost looks like the expected one (circles and lines).

2. I have noticed that every time the first seconds of the trajectory are very bad, and after ( sometimes ) it converges.

My settings :

=======================================
OPENVINS ON-MANIFOLD EKF IS STARTING
=======================================
ESTIMATOR PARAMETERS:
    - use_fej: 1
    - use_imuavg: 1
    - use_rk4int: 1
    - calib_cam_extrinsics: 1
    - calib_cam_intrinsics: 0
    - calib_cam_timeoffset: 0
    - max_clones: 8
    - max_slam: 40
    - max_slam_in_update: 25
    - max_msckf_in_update: 40
    - max_aruco: 1024
    - max_cameras: 1
    - feat_rep_msckf: GLOBAL_3D
    - feat_rep_slam: ANCHORED_INVERSE_DEPTH_SINGLE
    - feat_rep_aruco: ANCHORED_INVERSE_DEPTH_SINGLE
    - dt_slam_delay: 3.0
    - init_window_time: 2.00
    - init_imu_thresh: 1.00
    - zero_velocity_update: 1
    - zupt_max_velocity: 0.30
    - zupt_noise_multiplier: 50.00
    - zupt_max_disparity: 0.5000
    - zupt_only_at_beginning?: 0
    - record timing?: 0
    - record timing filepath: /tmp/traj_timing.txt
NOISE PARAMETERS:
    - gyroscope_noise_density: 0.013680
    - accelerometer_noise_density: 0.33151
    - gyroscope_random_walk: 0.0005509
    - accelerometer_random_walk: 0.000556
    Updater MSCKF Feats:
    - chi2_multipler: 2.0
    - sigma_pix: 2.00
    Updater SLAM Feats:
    - chi2_multipler: 1.0
    - sigma_pix: 1.00
    Updater ARUCO Tags:
    - chi2_multipler: 1.0
    - sigma_pix: 1.00
    Updater ZUPT:
    - chi2_multipler: 0.0
    - sigma_pix: 1.00
STATE PARAMETERS:
    - gravity_mag: 9.8100
    - gravity: 0.000, 0.000, 9.810
    - calib_camimu_dt: 0.0250
cam_0_fisheye:0
cam_0_wh:
640 x 480
cam_0_intrinsic(0:3):
502.164  500.82 325.656 236.818
cam_0_intrinsic(4:7):
  0.172458  -0.276475 -0.0227211 0.00209079
cam_0_extrinsic(0:3):
0.5 0.5 0.5 0.5
cam_0_extrinsic(4:6):
-0.05 0.041  0.03
T_C0toI:   // Manually set, almost same results with kalibr matrix
     0      0      1  -0.03
     1      0      0   0.05
     0      1      0 -0.041
     0      0      0      1

FEATURE TRACKING PARAMETERS:
    - use_klt: 1
    - use_stereo: 0
    - use_aruco: 0
    - downsize aruco: 1
    - downsize cameras: 0
    - use multi-threading: 1
    - num_pts: 150
    - fast threshold: 15
    - grid X by Y: 5 by 3
    - min px dist: 8
    - hist method: 1
    - knn ratio: 0.700
    - use mask?: 0
    - triangulate_1d: 0
    - refine_features: 1
    - max_runs: 5
    - init_lamda: 0.001
    - max_lamda: 10000000000.000
    - min_dx: 0.0000010
    - min_dcost: 0.0000010
    - lam_mult: 10.000
    - min_dist: 0.100
    - max_dist: 60.000
    - max_baseline: 40.000
    - max_cond_number: 10000.000

My bag : https://drive.google.com/file/d/1bCPfgME_Vmn9NT7hnT5SX-9irCFdLNsH/view?usp=sharing

Have you an idea to get a better trajectory ? Again, thank you so much for your works.

[WoosikLee2510]

• Is it your own dataset? If it seems to be related to your initialization, try setting a different init_imu_thresh. You can first set a large number and see when IMU measurement jumps up (when the system starts to move, check the value) and set the number.
• recommend calib_cam_intrinsics and calib_cam_timeoffset to be true unless you are sure about the calibration or system cannot handle more computation.

Try these first.

[goldbattle]

Hi, can you also try the latest master branch just released today? Can you be a bit more specific about what hardware you are using (links would be nice).

As Woosik mentioned it might be the initialization parameters which are used to detect when the sensor is picked up.

- init_window_time: 2.00
- init_imu_thresh: 1.00

2 seconds is likely too long here.

[ClementLASSENA]

Thank you a lot for your fast answers ! @WoosikLee2510 , it's my own dataset. I can make a new one if it can help. As you suggested I'll try other initialization settings. Turn on calib_cam_intrinsics and calib_cam_timeoffset

@goldbattle my setup in details : Software : Raspberry Pi 4B with Raspbian 10 Hardware :

1. Cam : Raspberry Pi Camera Module v2 https://www.digikey.ca/catalog/en/partgroup/raspberry-pi-camera-module-v2/63181?utm_adgroup=General&utm_source=google&utm_medium=cpc&utm_campaign=Dynamic%20Search_EN_RLSA&utm_term=&productid=&gclid=CjwKCAiA-9uNBhBTEiwAN3IlNGIg_Ejo8KEncctDeceGvvUkxJbIUs7gS4Y8HbDQBZmnS3IlPo7ygxoC8QEQAvD_BwE

2. IMU : Pololu MinIMU-9 v3 https://www.pololu.com/product/2468

I hope it helps. I'll try new settings and share results.

[ClementLASSENA]

Ok, Thanks to your advice I got a better results. Before to show them I have to answer to that :

Hi, can you also try the latest master branch just released today? Indeed I don't try. If you think it's very relevent in my case I can try to test it tomorrow (it's a little bit long on Raspberry PI ).

The improvment : I work on the initialisation parameters : ` - init_window_time: 0.20

• init_imu_thresh: 0.70 The beginning of the trajectory seems to be less azardous. Can you explain me theinit_window_time` parameter ?

I also work on the ZUPT parameter : ` - zupt_chi2_multipler : 0.00007

• zupt_max_velocity: 0.10
  • zupt_noise_multiplier: 50.00
  • zupt_max_disparity: 0.2000`

Can you explain me the zupt_chi2_multipler parameter too ? I don't really understdand him and his value seems weird. Finally I got this trajectory :

Pretty well even if sometimes it diverges, so thank you. However I got this new error :

What does it mean ? Thank you again.

Edit : I also turn on calib_cam_intrinsics and calib_cam_timeoffset.

[WoosikLee2510]

init_window_time is the length of window of IMU measurements you use for init : link zupt_chi2_multipler: it is a scaling factor for chi2 test threshold link. Lowering it will less allow zero velocity update. setting it 0.0007 should be almost the same as disabling zero velocity update. Also, zupt_noise_multiplier is (kind of) scaling the uncertainty of the zero velocity test, setting it a large number will more allow zero velocity test to pass, thus there will be more zero velocity update. setting it 50 seems very, very large and we generally recommend setting it 1~2.

The warning message is disabled now (not sure what version of OV you are using) but should come out from here: link this means your camera feature extraction is somewhat having trouble. Multiple reasons can cause this, such as bad calibration, your state estimation failure (or about to fly away), etc.

How were the calibration values after run? Were they calibrated to different values? did you try using the calibrated value as the initial guess?

[ClementLASSENA]

Hi, I tried lot of things. Replacing calibrated value as initial guess give me better results. So I tried to make a dataset where I walk in corridors ( square shape ).

As you can see it's not so bad. But my main issue is still there, if I launch twice the same bag with same parameters I got 2 results ( totally ) different. Same parameters and bag than last picture :

Also there is the drift and the divergent behaviour after one turn, or after a long period of time. Which parameters should I try to adjust to obtained better results?

Edit : I have noticed that the number of features used for the update is very low ( less than 8 per frame ) and sometimes equals to 0. When the trajectory start to fly away the number of features fall to 0.

Thank you so much again.

[goldbattle]

I would enable all calibration (pose, intrinsics, and time offset). Basically, I recommend you take the euroc_mav config and change the sensors to your own. https://github.com/rpng/open_vins/tree/master/config/euroc_mav

In terms of repeatability, if your computer can't process everything in realtime then you could be dropping frames which will give you random performance (I assume you are using the subscribe and not the serial launch files).

I also recommend you disable zero velocity update until the system is working well first. Also try using just the euroc mav IMU noises (or inflate them by one order) since these normally work pretty well all around.

If you post a zip of your config folder and an example bag we can try to discuss more specifics on what to edit.

[ClementLASSENA]

Hi, Sorry for the delay. After lot of research I found a (big) mistake in my IMU publisher node. Indeed the IMU frame was not right handed. After I have fix that the results are better and more regular. So I think we can close this issue. About synchronization between camera and IMU, is the algorithm deals with the IMU and Cam stamp ( in the header ) ? Tomorrow I will try to improve the results. If I encounter new difficulties and if you allow it I will create an other issue and I will share a zip with my bag and parameters. Thank you again !

[goldbattle]

Good catch, do you have a link to the driver you are using (or is it handwritten)?

The time offset is based on the camera and IMU ROS headers, yes. This is the same one that Kalibr estimates when you do your calibration. If you timestamp both with ros::now() then it should be fine if the delays are relatively constant.

[ClementLASSENA]

Yes it's an adapted version of this node : https://github.com/jeskesen/i2c_imu

But they is something that I don't understand on the IMU functioning : If I put the z axis in front of the ground, am I supposed to read 9.81 or -9.81 ?

After I thought it was 9.81 but finally the good answer seems to be -9.81 ! This result is counterintuitive for me. Have you an explanation " with the hand " ?

[goldbattle]

In terms of "right handed coordinates"? I would think of this picture: https://www.scratchapixel.com/lessons/mathematics-physics-for-computer-graphics/geometry/coordinate-systems

[ClementLASSENA]

Thank you for the link, however I know what mean right handed. My answer is, when I put the Z axis of the IMU in face of the ground, have I to read 9.81 or -9.81 on the z axis ?

Sorry I'm asking questions that don't have direct link with open_vins.

Edit : The answer is -9.81m/s^2. We can grossly interpret that as the IMU accelerates against the gravity to stay in place ( and doesn't fall ).

A more mathematical demonstration is to note that during free fall ( with the z axis in face of the ground ) we can read ( approximately ) 0 on the z axis. But in reality the IMU accelerates to the centre of the earth. So we have to correct this value by adding +g. So after correction during the fall we have acc_z_corrected = +g and in static condition we have acc_z_corrected = 0 = acc_z_raw + g. That is why in static conditions acc_z_raw = - g when z axis is pointing to the ground.

I hope this can help somebody !

[mzahana]

Hi @ClementLASSENA . I know this is an old post, but I wanted to ask how you could compile this package on RPi 4B. I have RPi 4B with 4GB RAM, and I am trying to compile it on ROS2. The compilation always gets stuck at the ov_msckf and the RPi freezes. I used the command colcon build --parallel-workers 1 --executor sequential --cmake-args -DCMAKE_BUILD_TYPE=Release to limit the paraller process and increased the swap size to 2GB, but it still freezes. Any hints are highly apreciated.

Thanks.

[mzahana]

After some searching, I was able to compile the pkg using the following command

MAKEFLAGS="-j1 -l1" colcon build --executor sequential --cmake-args -DCMAKE_BUILD_TYPE=Release

And it took 40 minutes.

[poudyalbot]

Sir how many fps is the system running.

On Mon, 25 Mar, 2024, 6:25 am Mohamed Abdelkader Zahana, < @.***> wrote:

After some searching, I was able to compile the pkg using the following command

MAKEFLAGS="-j1 -l1" colcon build --executor sequential --cmake-args -DCMAKE_BUILD_TYPE=Release

And it took 40 minutes.

— Reply to this email directly, view it on GitHub https://github.com/rpng/open_vins/issues/210#issuecomment-2017029810, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFIRO52PTZWK2HMIVYLBPCTYZ5YWXAVCNFSM5J7HPSK2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBRG4YDEOJYGEYA . You are receiving this because you are subscribed to this thread.Message ID: @.***>