facebookresearch / tacto

Simulator of vision-based tactile sensors.
MIT License
333 stars 75 forks source link

[Request/Support] Using Grasping Stability NN in real world #31

Open robertokcanale opened 2 years ago

robertokcanale commented 2 years ago

Hi @wx405557858, my team and I have bought the DIGIT sensors (directly from GelSight. I have read in your paper about the grasping stability NN that you trained. image

We intend to use the tactile feedback from the sensors to establish the stability of the grasp (we uese real-life YCB Dataset objects). We have some ideas in mind, but we would also test out the NN you trained in the real world. We would probably like to implement it within ROS, so getting anything from python should work. Could you kindly walk me through, at code level:

-How to run inference of the NN? -Would it be possible to have the script to run it and the weights? ( we would probably do a finetuning on top of it) -How/Where to provide the DIGIT tactile images as input? -What exactly is the provided output? Regards Roberto