Getting SMPL vertices from POINTCLOUD

[aakritiadhikari]

Hi,

Great work! After some modifications code works! making code work in the python2.7 environment was a little bit problematic!

I am wondering about the way you collected SMPL vertices from the point cloud and the way you did your background subtraction. I read the paper and I saw you downsampled PC but didn't get much information about the process. It would be great if you can reply.

[henryclever]

Glad you were able to get it working!

First I used a fairly conservative height cutoff that removed many of the points below a specific height, but left others there due to folds or otherwise relatively high locations on the mattress. If I made the cutoff too high it would cutoff points from the hands or other body parts where the surface isn't much higher than the mattress. Then I used some processing functions from open3d to remove "islands" of point clusters. This left the points representing hands and arms and other "low" parts of the body intact because they were connected to the rest of the body, but eliminated most everything from the mattress and sheets.

http://www.open3d.org/docs/latest/tutorial/Advanced/pointcloud_outlier_removal.html

I went through it manually to make sure, and i think there were a couple others I had to cut out remaining point clusters by hand, but it was only a couple.

-Henry

[aakritiadhikari]

Thank you for your response.

I tried a similar approach and just took the pixels inside a predefined mattress size, but, due to folds happening between hands and leg, thresholding is really difficult. For example in this picture (reference RGB picture), there are uneven variations due to which some parts of the leg go missing even with a fairly conservative cutoff and removing "islands" of point clusters. (I will try to look into it manually but I am hoping to automate it).

However, my question is: Is there any reason why you chose to specifically wear a black bodysuit and tie hair with bands? I also observed in my own dataset that the head putting pressure on the pressure mat, creates the head to go deeper, causing the mattress to rise unevenly compared to other body parts. So, was it helpful during post-processing with the bodysuit?

Reference_picture:

[henryclever]

That is a tricky fold in the mat to segment.

I recall for my work that I used thick tape and velcro to secure the mattress on all 4 edges and throughout the middle, and a tight fitted sheet on top. Some of the tape left residue on the mat, but oh well. I don't recall having folds as large as what is shown in your photo.

Regarding the body suit and the hair cap - I wanted to have clean color contrast between the background and foreground for RGB clarity in the qualitative results. The suit is also good because it is more form fitting and reduces noise from baggy clothing that could otherwise adversely affect the optimization process I used to fit SMPL to the point cloud for ground truth data. Moreover, the depth camera sensor can respond differently to different fabrics. Some might show a few more or fewer inches depth -- which is material dependent. The body suit clears up these potential discrepancies, and the cap does the same for the hair. Depth sensors don't work well with human hair in my experience.

Hope this helps! Good luck. -Henry