question about a jacobian

[fushi219]

Hi, thanks for your beautiful work! I got a question about the jacobian at: https://github.com/SpectacularAI/HybVIO/blob/main/src/odometry/triangulation.cpp#L963

In my understanding, its about such a question: P_{point_incamera} = R^{cam}{world} * P_{point_inworld} and R = R(q). then we want to get the jacobian of d(P{point_in_camera}) / d (q), why this jacobian is related to camera and imu baseline parameter?

I am trying to make equirectangular images + IMU available on your code, do you think this is a practicable idea?

[oseiskar]

Hello. Thank you!

The "baseline" (which is defined here) appears because the elements $(p, q)$ in the pose trail are defined as the poses of the IMU sensor. The "baseline" in that equation is related to how the pose of a particular camera is translated in case of pure rotations around the IMU.

It should definitely be possible to extend the code to support equirectangular images, but it may be rather complicated in practice and we do not really offer any support for that. One potential pitfall is the use of "normalized image coordinates" in some of the EKF update equations, which probably won't work with equirectangular data. So you would have to reformulate those equations to use bearing vectors instead.

We also cannot guarantee to offer any kind of practical support for this, unfortunately.

[fushi219]

Thanks a lot! It's my fault that i thought the pose is defined in camera as it is called as 'CameraPose'. Now, i got it. Thanks you！！

Besides, i have done the work you talked about as i found the norimalized coordinates used in hybridvio can't express points 'behind' equirectangular 'lens', where bearing vector could. As soon as i finished my work, i would like to show you the result.

Finally, i wish to ask for advice about the calibration of extrinsics between equirectangular 'lens' and imu(i got the data from a gopro-max). I have two ideas. One is to modify kalibr to support such calibration. The other is to trans equirectangular into cubemaps and use the front pinhole images to calibrate extrinsics between lens and imu. But i wonder if the result make sense in such method？

Thank for your beautiful hybrid-vio！It actually works better than orbslam3, vins-fusion, okvis on my insta360 camera in mono mode.

[fushi219]

Thanks a lot!  It's my fault that i thought the pose is defined in camera as it is called as 'CameraPose'. Now, i got it. Thanks you！！

Besides, i have done the work you talked about as i found the norimalized coordinates used in hybridvio can't express points  'behind' equirectangular 'lens', where bearing vector could. As soon as i finished my work, i would like to show you the result.

Finally, i wish to ask for advice about the calibration of extrinsics between equirectangular  'lens' and imu(i got the data from a gopro-max). I have two ideas. One is to modify kalibr to support such calibration. The other is to trans equirectangular into cubemaps and use the front pinhole images to calibrate extrinsics between lens and imu. But i wonder if the result make sense in such method？

Thank for your beautiful hybrid-vio！It actually  works better than orbslam3, vins-fusion, okvis on my insta360 camera in mono mode.

---Original---  From: "Otto @.> Date: Wed, Jun 15, 2022 23:39 PM To: @.>; Cc: @.**@.>; Subject: Re: [SpectacularAI/HybVIO] question about a jacobian (Issue #23)

Hello. Thank you!

The "baseline" (which is defined here) appears because the elements $(p, q)$ in the pose trail are defined as the poses of the IMU sensor. The "baseline" in that equation is related to how the pose of a particular camera is translated in case of pure rotations around the IMU.

It should definitely be possible to extend the code to support equirectangular images, but it may be rather complicated in practice and we do not really offer any support for that. One potential pitfall is the use of "normalized image coordinates" in some of the EKF update equations, which probably won't work with equirectangular data. So you would have to reformulate those equations to use bearing vectors instead.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>