A question on the 'RBSolver' class

[Learner209]

Hello! That's great work out there! I have a little question after reading and running your code: The RBSolver was rebuilt from scratch during every exploration iteration, and every time the RBSolver is in training, it will start optimizing the parameter Tc_c2b from the initial pose obtained from running PVnet(as in the code: self.init_Tc_c2b), so in other words, the optimization of the camera pose doesn't use the training output in the previous iteraitons? Am I missing something? Or it is supposed to be in this way?

[ootts]

Hi! Thanks for your attention! Yes, for each iteration, the pose is optimized from the init pose instead of the result of last iteration. We find the latter one gives a lower performance since the rendering based optimization is highly non-convex problem and thus leading to local optim. Optimizing from the init pose gives a higher probability to converge to the correct global optim.

[Learner209]

I got it! Thanks for your quick reply!

[Learner209]

Hello again! I am a member from the Robotflow AI team from SJTU. My supervisor is Cewu Lu. I recently adapted your code to the Franka robot in our lab while maintaining the Xarm part and found the result to be very accurate. So, are you interested in a PR maybe?

[ootts]

Nice! Please add my WeChat clhfsky. I would also like to discuss with you about releasing models trained on other robots!