Misunderstanding of gel_min_depth and max_depth config parameters

[Jcastanyo]

Hi! First of all I wanted to congratulate you for your work, it is really amazing.

I wonder what the gel_min_depth and max_depth config parameters mean. In digit.yaml, I can see your values of ~20mm, but I don't really understand how the gel is able to deform this much. The thickness of the sensor is around 2-3 mm, so it seems impossible to deform 20mm.

If you could guide me with these two parameters, I would appreciate it.

Thanks in advance. Julio.

[vocdex]

Hi Julio! It's been a while but if I remember correctly, gel_min_depth and max_depth are not the actual deformation of the gel. They are the distances from camera (bottom of the sensor) to deformed and undeformed gel. You can first try to measure these distances. Hope it's helpful.

[Jcastanyo]

Hi, thanks for your quick answer.

I've managed to train the MLP and test it with a few images from my DIGIT sensor. However, the depth estimation is not correct when there is no contact applied to the sensor. Maybe you have experienced this issue and you can help or guide me to improve the quality of the depth estimation.

I attach the training images, the cvs file, the digit.yaml, and a few examples of the depth estimation. In the examples you will see that the model estimates an initial depth but there is no contact applied in that area.

images.zip

annotate.csv

[vocdex]

That seems strange to me. It's been a while since I calibrated the sensors myself and I no longer have access to one, so I cannot test it on my own. I would suggest doing another annotation and playing around with the gel parameters in the config file. Also, I noticed you annotated more around 700 images. Usually, for me, abour 30-40 images were enough to get good depth estimation. So, maybe, please try to use only 40 images in training the MLP model.

[Jcastanyo]

Hi, I am using the label_data.py script to label the images. I was annotating several points of the same circunference, that is why there were lots of rows inside of the csv file. Now, I am labelling only the center and one point of the circunference. Is that the right way to do it? I am getting the same depth estimation. Which gel parameters can be changed? Thanks in advance.

[Jcastanyo]

Hi again,

Now I am getting better results. I think the problem is that the images need to contain contact regions in different positions of the gel. I will try to record more images with more variety of contact location and size to see if I can improve the results.

Now, I am trying to understand the mathematical equations to obtain surface normals from pixel values from these lines:

dist = np.sqrt((x - center_x) 2 + (y - center_y) 2) ang_xz = math.acos(dist / radius) ang_xy = math.atan2(y - center_y, x - center_x) nx = math.cos(ang_xz) math.cos(ang_xy) ny = math.cos(ang_xz) math.sin(ang_xy) nz = math.sin(ang_xz)

but I am struggling a lot. I have also read the gelsight article but I can't see how to get these values.