Open matpalm opened 5 years ago
It sounds like the grasping is just too fine grained to be captured by the embedding (although I am still at the stage of trying to run the code by myself). I was wondering if you can try training the TCN not on the entire scene, but on a close up of the object only. And then let RL (policy search would be my choice) to reward the robot for reproducing the observed grasp, regardless of the arm pose. In the end, if the task is to grasp, addressing the arm pose may be secondary?
the entire point of me starting this project was to try to train a grasping model but i haven't got there yet o_O as things are i get the feeling this embedding might be capturing something about the arm but not really anything about the objects. still an open question...