How can I delete or deactivate an edge during the optimization

[yangliu9527]

After adding vertices and edges to the optimizer, the optimizer is initialized and then optimization can be performed iteratively. Such as: optimizer.initializeOptimization(int level) optimizer.optimize(int iteration_number)

However, during the optimization I want to deactivate some of the edges according to the computed error. Actually, the defined residual is a pixel value of a distance transform (DT) image. Therefore, if the projected pixel of a landmark is out of the DT image, the corresponding edge should be removed and will not participate into the next iteration. (The situation is very similar to the direct SLAM method, which minimizes the photometric error. During the iteration process, the projected landmark may be out of the image)

I have considered two ways to achieve my purpose, but I don't know if it is suitable or elegant.

WAY 1: I know after or before the optimization, I can deactivate an edge via setting its level to 1, such as edge.setLevel(1). But I wonder whether I can deactivate an edge just during the optimization. So, can I change the level of an edge in the function of computing error, for example: void computeError() { Eigen::Vector2d projected_pixel = (K*Tcw*Pw).block<2,1>(0,0); if(projected_pixel[0]<=0 || projected_pixel[0]>Image.cols || projected_pixel[1]<=0 || projected_pixel[1]>Image.rows) setLevel(1); _error = Image.ptr<double>(row)[col]; }

WAY 2: There is a simple way that is distributing N iterations into N loops. In every loop I only perform just one iteration and then I check if every landmark can be projected inside the image via the optimized pose. After setting the level of invalid edges to 1, then the next iteration is performed in the next loop. But you know, this way is not elegant, it requires many repeated loops and initialization, which cause too much computation burden.

For this kind of problems, do you have any good suggestions?

[RainerKuemmerle]

Let's first talk about 2: You can achieve this by virtual bool initializeOptimization(HyperGraph::EdgeSet& eset); without changing the level but initialize directly with the edges you want for one or multiple iterations. Effect will be the same in the end.

1: In the example you stated above, you could potentially also set the error to 0. However, this might lead to issues with your Jacobian or has to be considered there as well. Might it be that the optimization problem ends up with a lower rank.

To account for that you may want to look at RobustKernel to handle outliers. We provide several implementations https://github.com/RainerKuemmerle/g2o/blob/master/g2o/core/robust_kernel_impl.h

In general I am, however, not entirely sure if this is really the best approach. If you happen to have a valid and correct measurement on your image border, you would be likely to remove it from subsequent optimization runs just because it is reprojected outside of your lookup data structure. In your example, of a pin-hole camera, one can still compute "valid" error value for a point being reprojected just outside the image plane of the camera at use. It can just be the 2D distance between measured and reprojected pixel.

[yangliu9527]

Let's first talk about 2: You can achieve this by virtual bool initializeOptimization(HyperGraph::EdgeSet& eset); without changing the level but initialize directly with the edges you want for one or multiple iterations. Effect will be the same in the end.

1: In the example you stated above, you could potentially also set the error to 0. However, this might lead to issues with your Jacobian or has to be considered there as well. Might it be that the optimization problem ends up with a lower rank.

To account for that you may want to look at RobustKernel to handle outliers. We provide several implementations https://github.com/RainerKuemmerle/g2o/blob/master/g2o/core/robust_kernel_impl.h

In general I am, however, not entirely sure if this is really the best approach. If you happen to have a valid and correct measurement on your image border, you would be likely to remove it from subsequent optimization runs just because it is reprojected outside of your lookup data structure. In your example, of a pin-hole camera, one can still compute "valid" error value for a point being reprojected just outside the image plane of the camera at use. It can just be the 2D distance between measured and reprojected pixel.

Thanks for your timely reply.

1. For your first suggestion, the problem is that I still have to check every edge in each loop to decide whether it will be added in the EdgeSet for the next iteration, that's why it costs time. But I appreciate your instruction because it's a new usage for me.
2. For your second suggestion to set the error to 0, I think it may be a feasible way to solve the error computing problem, but as you have mentioned, there is still a problem in Jacobian caculation. And I should explain the definition of the residual in more details. The residual here is not the re-projection error, but more similar to the one-dimensional photometric error which is defined as the difference between the grayscale value of the projected pixel and the reference grayscale value. That means both the residual and Jacobian calculation will depend on the image (the residual involves the grayscale value difference and the Jacobian involves the gradient of the grayscale). That's why the problem arises. Have you met some others' problem about developing direct SLAM method (e.g. LSD-SLAM, DSO) using g2o as the back-end? I will try the WAY2 first because it seems like more avaliable.

[yangliu9527]

The WAY1 works. Thanks a lot!