search

introlab / rtabmap

RTAB-Map library and standalone application
https://introlab.github.io/rtabmap
Other
2.72k stars 775 forks source link

IMU Local Transform of OAK Cameras #1038

Closed borongyuan closed 1 year ago

borongyuan commented 1 year ago

Hi, I'm trying to update the IMU local transform of OAK cameras as planned in https://github.com/introlab/rtabmap/pull/1034#issuecomment-1554009619. Since the official factory calibration does not currently include IMU extrinsics, I am measuring through PCB files. But the IC mounting position on the PCB is not enough to determine the coordinate system of the IMU. Because axis remapping may be applied in the firmware. We know that OAK cameras manufactured between Q2 2021 and Q2 2023 use BMI270, not BNO085. According to the documentation, BMI270 and BNO085 have different default axes orientation as shown below. But according to my test, they should have applied axis remapping to BMI270, so that the axes orientation is consistent with BNO085. BNO085 BMI270 If this is correct, then we can just modify the code to check the camera model instead of the IMU model. However, I don't have all types of OAK cameras on hand, so I hope you can help provide verification results. I currently have OAK-D S2 and OAK-D Pro W and they share the same PCB design (DM9098). I also have an OAK-D PCBA (BW1098), but it's a crowdfunded Early Bird model that doesn't have an integrated IMU, and doesn't even have any factory calibration. I should try OAK-D W PoE (NG9097) later if I have the chance. For the first generation of OAK-D, I noticed that you fixed the local transform in Commit 4321c3040ada190d019a9ac0e969f4031d2ebb89. The modified axes orientation is consistent with what I see in this post (x->down, y->left, z->forward). Can you confirm that this is currently unchanged? For DM9098, since the IMU is changed to the opposite side of the PCB, the axes orientation becomes x->down, y->right, z->backward.

borongyuan commented 1 year ago

I found the Look Up Table for boardName from here.

matlabbe commented 1 year ago

I did update you PR for original OAK-D (with IMU) in this commit: https://github.com/introlab/rtabmap/commit/5b1c9e7233b7a820cea0623dbcbf5a4dc7fee521 (now working on my side, otherwise the board was not recognized).

For the other boards, I cannot really say. I updated the warning with the name of the board.

borongyuan commented 1 year ago

So your boardName is also BW1098OBC and it has IMU on board? Looks like only I am a victim of the early product😅

matlabbe commented 1 year ago

Do you know if with latest depthai sdk this function doesn't crash anymore? https://github.com/introlab/rtabmap/blob/1b67d6a86a9b0585dec48da2e27de9f8f6042dcf/corelib/src/camera/CameraDepthAI.cpp#L273-L274

Otherwise I will close this issue as I confirmed is working with OAK-D (the one with IMU!).

borongyuan commented 1 year ago

This should not be an SDK issue. At present all OAK cameras do not have IMU extrinsics calibration before shipment. This should be their repository for factory calibration. There is no part related to IMU calibration. Actually they should put the board design data in eeprom. This is accurate enough for most applications. I am currently preparing to purchase OAK-D Pro W PoE. I think we can wait until the PoE board information is updated before closing this issue.