search

uzh-rpg / rpg_monocular_pose_estimator

A monocular pose estimation system based on infrared LEDs
GNU General Public License v3.0
134 stars 78 forks source link

Unable to resolve pose when one led is occluded #15

Open cmas1 opened 5 years ago

cmas1 commented 5 years ago

Hi,

I am testing this node in simulation, on VREP. I have created a scene with 6 markers and one camera. The markers are not coplanar and well spaced The markers are now simple spheres and I added some filters on the camera in order to mimc the IR emitters and IR camera.

I set up the position of the markers in the yaml file and lanched the node. What happens is that when all 6 markers are visible, the pose estimation works very well. However, as soon as I occlude any of the 6 markers the pose estimation fails and I get the warning "Unabel to resolve a pose".

I verified that the blobs are detected correctly. I believe my problem is at a later step, perhaps in the correspondence check. I have also tried to force and always reinitialize the estimation, and the problem persists. So it looks like is is not limited to neither the brute force correspondence check nor the correspondence check from the predicted pose.

Here are the parameters I used in the launch file param name= "threshold_value" value = "140" param name= "gaussian_sigma" value = "0.6" param name= "min_blob_area" value = "60" param name= "max_blob_area" value = "2000" param name= "max_width_height_distortion" value = "0.7" param name= "max_circular_distortion" value = "0.7" param name= "back_projection_pixel_tolerance" value = "5" param name= "nearest_neighbour_pixel_tolerance" value = "70" param name= "certainty_threshold" value = "0.75" param name= "valid_correspondence_threshold" value = "0.7" param name= "roi_border_thickness" value="100"

Any suggestion at what could be the cause or to what part of the algorithm I should focus looking at?

cmas1 commented 5 years ago

Ok, I dug a little deeper in the code. I found that one problem occurred when looking for correspondences in the histogram.

In PoseEstimator::correspondencesFromHistogram there is a check if (max_value < histogramthreshold) break;

What was happening is that the histogramthreshold was set at 20, but the values in the histogram would drop below 15 whenever a led is not detected. In turns this I think depends on some hyperparameters such as tolerances in the pixels and thresholds in the correspondences. I improved my results by fiddling with the following parameters: back_projection_pixel_tolerance nearest_neighbour_pixel_tolerance certainty_threshold valid_correspondence_threshold

Ultimately, I can also modify the histogram_threshold directly, however hardcoding a threshold value like that might not be the best idea.

I still need to verify what happens when looking for correspondences not in the histogram, but from the prediction, so that point is still open. Anyway I expect it to be related to the choice of parameters and perhaps some other threshold.