laxnpander / OpenREALM

OpenREALM is a pipeline for real-time aerial mapping utilizing visual SLAM and 3D reconstruction frameworks.
GNU Lesser General Public License v2.1
432 stars 111 forks source link

Why do we need a gimbal-stabilized camera? #107

Open marleyshan21 opened 2 months ago

marleyshan21 commented 2 months ago

Hi @laxnpander .

The Readme has the following statement: "The pipeline is designed for multirotor systems with downlooking 2-axis gimbal stabilized camera and alignment of the magnetic heading of the UAV with the image's negative y-axis."

It makes sense for gnss only versions since we can only project it downwards. But does this assumption extend to vslam based stitches too? Arent we getting 3D rotation from the VSLAM?

I am trying to understand which part of the codebase gets affected when we break this assumption (for vslam versions).

marleyshan21 commented 2 months ago

So, it is related to the umeyama initialization right?

Screenshot from 2024-04-18 13-21-33

This image from your thesis talks about the issue with using the normal umeyama as multiple visual frames, with different rotations, but with the same translation will provide the same result. This is an intrinsic issue with umeyama in itself, since it just takes the positional coordinates and provides frame-frame transformation. You are using virtual points to influence it to take the z down direction.

Now provided I can get the roll, pitch and yaw of the drone in the gravity aligned world frame, and that I want to allow the drone to roll and pitch, how can i make sure that my umeyama initialization wont get affected?

Any suggestions or ideas?