search

ami-iit / kindyn-vio

Development on hold.
BSD 3-Clause "New" or "Revised" License
13 stars 1 forks source link

Implement lines tracking using OpenCV LineSegmentDetector #11

Closed prashanthr05 closed 3 years ago

prashanthr05 commented 3 years ago

I had already implemented reference code in Matlab. I need to find it and make it into proper classes.

prashanthr05 commented 3 years ago

Check

prashanthr05 commented 3 years ago

OpenCV (>4.0.0) has problems with LineSegmentDetector implementation due to licensing issues.

See https://github.com/opencv/opencv/issues/14576.

cc @traversaro

prashanthr05 commented 3 years ago

In the projects PL-VINS and rubengooj/stvo-pl which use OpenCV implementation of Line Segment Detector, the corresponding source code from OpenCV Contirb has been added as a 3rd party library with BSD license.

prashanthr05 commented 3 years ago

We may first try to use OpenCV's FastLineDetector along with BinaryDescriptor and BinaryDescriptorMatcher class to see if we can have a desired outcome for lines tracking, before thinking about tackling the Line Segment Detector (LSD) issue. However to use FLD with BDM, we might need KeyLines data which is not provided completely by FLD.

(Also BinaryDescriptorMatcher comes within the same module line_descriptor, so it might be tricky to use any one of these).

Update: One work-around would be to use FLD detector to detect lines and use KLT on the endpoints for tracking the lines.

prashanthr05 commented 3 years ago

Update: One work-around would be to use FLD detector to detect lines and use KLT on the endpoints for tracking the lines.

I implemented this logic (quite similar to the PointsTracker class described in https://github.com/dic-iit/kindyn-vio/issues/10#issuecomment-885442260).

The results are shown in the following video. It shows the line features tracked across multiple frames, where each detected line is described as L<feature-id>, <frame-track-count>, where L signifies a line feature . For example L3, 10 means the line feature was assigned with the ID 3 and has been tracked for 10 consecutive frames. The color of the marker changes from green to red depending on how long it has been tracked.

https://user-images.githubusercontent.com/6506093/127493229-35344fd4-a09e-428a-b3d5-ec93f3324d6a.mp4