Problems when putting color, rgb, gyro and acceleration sensor into one pipeline

[vartid-ak]

Hi,

Thank you for preparing this wrapper. It helped a lot.

When we used an Orbbec Astra2 camera and this C# wrapper in our project we faced to the next situation, which could be related to the OrbbecSDK itself, and not the C# wrapper, but as we used this wrapper with the included dll-s I decided to put here instead putting it to the OrbbecSDK page.

When we setup a pipeline with color, depth, gyro and acceleration sensors than the framerate we measure only around the half of the set one. 13-17 instead of 30, 6-8 instead of 15 and 2-3 instead of 5 FPS and it is not stable. If we set 30 FPS for color, and 15 fps for depth sensors, then we get stable 15 FPS. (in vice-versa also not stable)

We implemented a workaround, by separating the gyro and acceleration sensor into another pipeline on a different thread and implementing our own logic to find the closest gyro and acceleration frame for the synced color-depth frame pairs. (We just need to know if the camera was stable when the image was shot) With this workaround we can get stable 30 FPS synced information from color, depth, gyro and acceleration sensors at once.

If we used dlls from the latest SDK, the result was the same.

Do we missed something in the documentation and we should not put all sensors into one pipeline, or have we found something to be improved?

Thank you. András

[zhonghong322]

We recommend using a separate pipeline for the video frames (depth, IR, color) and another pipeline for the IMU (acceleration and gyro) because the IMU has a high frequency. If both are acquired through a single pipeline, the frameset obtained may not contain all frames simultaneously.

[vartid-ak]

Thank you for the clarification and the quick response 🥇

So, what we thought was a workaround is actually the suggested solution. :-)