search

PeterFWS / Structure-PLP-SLAM

[ICRA'23] The official Implementation of "Structure PLP-SLAM: Efficient Sparse Mapping and Localization using Point, Line and Plane for Monocular, RGB-D and Stereo Cameras"
GNU General Public License v3.0
394 stars 59 forks source link

Puzzled at using partial_occlusion for line projection #24

Closed C-H-Chien closed 2 months ago

C-H-Chien commented 2 months ago

Hi,

When projecting 3D lines to 2D images, to find a support from image observation, the code (i) first project two end points of the line to the image, and then (ii) find whether the reprojected end points are visible in the image space. If one of the end point is invisible, then the code checks the visibility of the mid-point of the line. If it is visible, partial_occlusion is set true. (see code starting this line)

However, later on the code does not use this partial_occlusion; instead, the two end points are still used to find 2D line support (see code here).

I am puzzled: what is the need of using partial_occlusion?

PeterFWS commented 2 months ago

Hi,

Thank you for reviewing the code! it is been a while.. :)

(1) I implemented this way in the projection::match_current_and_last_frames_line you mentioned, was because: when projecting 3D lines onto 2D images, it is important not to discard line candidates simply because one endpoint falls outside the image frame. A partially occluded line does not affect the minimization of line re-projection error in Bundle Adjustment, as the line segment can shift along its direction without altering the re-projection error.

Therefore I assume we should keep a 3D line if at least it has one end point and its middle point is visible in a certain image.

(2) as for the code get_keylines_in_cell where partial_occlusion is not used: if I remember right there are sometimes you need to find as many matched lines as possible somewhere in the tracking module. :)

Important Remark: We should rely on the robustness of BA to filter out any outliers of 3D-2D correspondences in optimization. Therefore the implementation should be just fine.

If you are trying to improve the implementation, do not need to take any part of the code for granted. Some of the implementations were based on heuristics. Just deactivate some parts of the code and see what may happen.

Thank you!

Best regards,

C-H-Chien commented 2 months ago

Thank you! I agree with you that we should not rule out lines of which at least one of the endpoints is within the image boundary. And I also agree that not using partial_occlusion does not have impact on the results.

Some of the implementations were based on heuristics. Just deactivate some parts of the code and see what may happen.

This makes sense to me. Thank you again for the reply!