search

vocdex / digit-depth

Package for DIGIT tactile sensor
26 stars 12 forks source link

Color to normals #3

Closed Jcastanyo closed 1 year ago

Jcastanyo commented 1 year ago

Hi! I am Julio again, I create this issue to ask you about the normals. How do you get the normals from the images for the ground truth? Thanks in advance!

vocdex commented 1 year ago

To obtain normals, we use a small sphere (D=6mm) and press it on DIGIT 40-50 times. We use sphere because it is possible to get ground truth normals for the sphere due to its geometry. Please refer to this script for the implementation. To understand logic behind it, please check section 3.5 of GelSight Paper In case you will be using this repo, you can label sphere radius and center based on /scripts/label_data.py and after this, you can simply execute create_image_dataset.py to obtain folders /datasets/A, datasets/B with ground truth normal images for spheres. Hope it was helpful.

Jcastanyo commented 1 year ago

Thanks!! I have one more question. If you train a neural network to calculate the normals, but the training set only contains normals from the sphere, It will generalize to other types of geometries? Thank you very much for your help!

vocdex commented 1 year ago

Yes, it will generalize quite well because we are only using sphere to learn a general relationship between RGB intensity and normals. Please refer to this and this papers' relevant sections for more info on generalization.

Jcastanyo commented 1 year ago

Ok, I will take a look at these papers. Thank you again for your attention and help!