Error of incorrect eigen matrix dimensions

[TheGoblinTechies]

Hi - Thanks for your nice tracking package. I have encountered this problem, the error information is here:

I have checked other issues to correct the 2D skeleton problem. But this error seems to be "incorrect eigen matrix dimensions" due to my limited C++ experience. This will only happen when the DLO moves a bit fast or interact with the external object (either by pushing the DLO or occluding the DLO from the Realsense, like my hand occludes the DLO). Would you know the specific reason and how to solve it? Many Thanks!

PS: Though this is not related to software or hardware, but I'll list them here: Ubuntu 20.04, ROS Noetic, Realsense D435. I have used the color picker to get the HSV filter range. I didn't use the docker, but package of C++ like eigen should be the same version in TrackDLO.

Zhaole

[hollydinkel]

Hi @TheGoblinTechies! I am glad to see you are using TrackDLO, and thank you for raising this issue!

I believe this issue is due to not enough of the object being visible. TrackDLO is able to track under significant occlusion, but when the object is almost fully occluded it loses track. If there are only a few points that define the object's raw pointcloud, it will be very difficult for the tracking steps to work.

We also show that TrackDLO cannot track the object in the event both tips are occluded in the supplementary video.

You can modify the launch file to auto-respawn the node if it fails like this:

// other TrackDLO parameters
<arg name="respawn" default="true"/>

<node name="trackdlo" pkg="trackdlo" type="trackdlo" output="$(arg output)" respawn="$(arg respawn)">
     // other parameters
</node>

<node name="init_tracker" pkg="trackdlo" type="initialize.py" output="$(arg output)" respawn="$(arg respawn)">
     // other parameters
</node>

Of course, if you find solutions to these issues yourself, we invite you to open a pull request! Contributions to this work are appreciated and encouraged! 😊

[TheGoblinTechies]

Hi Hollydinkel,

Thanks a lot for your quick response.

I have got through the code and noticed this problem happens at the function "MatrixXd trackdlo::calc_LLE_weights (int k, MatrixXd X)" in trackdlo.cpp, however it would be somehow difficult to debug the specific reason (https://github.com/RMDLO/trackdlo/blob/master/trackdlo/src/trackdlo.cpp#L119)

Another problem is after doing the HSV filtering, even there are almost no false positive pixels according to color_picker.py, I can still observe unusually much more isolated points in the point cloud (I assume the point cloud shown in rviz is already filtered), especially introduced by the robot end effector which often has the contact with the cable. I suspect too many points that don't belong to the cable leads to this problem.

This problem can be easier to solve by using a more contrast colored rope, although I am still a bit puzzled by the mentioned two problems.

Zhaole

[jingyi-xiang]

Hi Zhaole,

I'm not exactly sure why calc_LLE_weights would cause issues, but LLE (Locally Linear Embedding) is actually not used in the trackdlo algorithm. I included that function in the code only because other baseline methods use locally linear embedding. You could try to comment out lines 236, 237, 396 - 402, set include_lle to false, then run the code again. This should remove the computation of matrices L and E, hence the use of calc_lle_weights.

For color thresholding, trackdlo is indeed sensitive to the quality of segmented point clouds. A quick fix I sometimes use is to set a threshold for the point cloud distance and filter out the points outside of the threshold.

Jingyi

[TheGoblinTechies]

Hi Zhaole,

I'm not exactly sure why calc_LLE_weights would cause issues, but LLE (Locally Linear Embedding) is actually not used in the trackdlo algorithm. I included that function in the code only because other baseline methods use locally linear embedding. You could try to comment out lines 236, 237, 396 - 402, set include_lle to false, then run the code again. This should remove the computation of matrices L and E, hence the use of calc_lle_weights.

For color thresholding, trackdlo is indeed sensitive to the quality of segmented point clouds. A quick fix I sometimes use is to set a threshold for the point cloud distance and filter out the points outside of the threshold.

Jingyi

Hi Jingyi,

I didn't change the code but improve the tracking performance in a slightly more unstructured environment by applying a erosion (kernel size = 3 is enough) to the segmented mask to reduce random noise caused by imperfect HSV segmentation. I would suggest using a real-time visualization for HSV filter parameters selection rather than a single image since the realsense's RGB camera is of low quality and introduce much random noise, so the selected parameters from a single image might not be robust during the tracking.

Sincerely, Zhaole