process SMPL-X meshes

[davidpagnon]

I finally took the time to do the mapping between SMPL-X and SMPL+H points, and did a bit of tweaking for the script to accept the optional flag --smplx (see this issue).

I suppose the little shift is likely due to Mosh++ not fitting SMPL+H and SMPL-X meshes the exact same way.

I also added a flag --no_confirm, because in case you want to batch process multiple data, it becomes a bit annoying to manually input confirmation whenever you use a different marker dictionary or model.

It works but I'm still concerned: while the height is quasi-identical, the person's mass calculated from SMPL-X is much smaller than the one of SMPL+H (sometimes up to 20 kg). I haven't figured out why the volumes are so different yet. Do you have any idea why this could be?

For reference, here is the code for calculating mass:

    def compute_mass(self):
        ''' Computes the mass from volume and average body density
        Code adapted from Lea Muller (lea.muller@tuebingen.mpg.de), https://github.com/muelea/shapy/tree/master/measurements, 
        '''
        x = self.triangles[:, :, :, 0]
        y = self.triangles[:, :, :, 1]
        z = self.triangles[:, :, :, 2]
        volume = (
            -x[:, :, 2] * y[:, :, 1] * z[:, :, 0] +
            x[:, :, 1] * y[:, :, 2] * z[:, :, 0] +
            x[:, :, 2] * y[:, :, 0] * z[:, :, 1] -
            x[:, :, 0] * y[:, :, 2] * z[:, :, 1] -
            x[:, :, 1] * y[:, :, 0] * z[:, :, 2] +
            x[:, :, 0] * y[:, :, 1] * z[:, :, 2]
        ).sum(dim=1).abs() / 6.0
        mass = volume * self.DENSITY
        return mass.item()

[MarilynKeller]

Hi David,

Thanks so much for looking into this, the markers conversion seems to work great!

For the hands, maybe forcing the hand pose parameters of SMPLX to zero can help.

About the height computation You said the height values looks alright but I am surprised as you did not modified smpl_measurement_vertices.yaml. Maybe it works "by chance": Indeed, the function that computes the body height uses SMPL vertex indices corresponding to the top of the head and the bottom of the heals (this line). These vertex indices are loaded from this file. So if you are considering SMPLX mesh instead of SMPL, these vertex indices should probably be adapted. SMPLX has more vertices than SMPL so their indexing is different.

For the weight This function should be agnostic to the mesh topology so it does not explain such a huge difference. Although I took this function from SHAPY and I did not dig in how it works exactly (found some references here) You say you get different weight using SMPL and SMPLX. Do you mean you passed SMPL and SMPLX meshes of the same person and got different weight values? How did you get this pair of meshes, did you check visually that they represented the same body shape?

[davidpagnon]

Hi,

Yes, you are right about the smpl_measurement_vertices.yaml, I somehow missed it! I'll fix it today.

You say you get different weights using SMPL and SMPLX. Do you mean you passed SMPL and SMPLX meshes of the same person and got different weight values?

Yes, exactly. I got meshes from the BEDLAM dataset. You can choose to download them either as SMPL or as SMPL-X ones, so I did both. They look similar, and the output markers are also quite similar so I don't believe this is where the difference in volume comes from.

Chat-GPT helped me figure out how the volume calculation works:

The volume calculation for a mesh here is based on the signed volume of the tetrahedrons formed by the origin and each triangle in the mesh. This method is an extension of the formula for the volume of a tetrahedron, applied to a triangular mesh. Here's a step-by-step explanation:

1. Extract Coordinates: The x, y, and z variables extract the coordinates of the vertices of each triangle from the self.triangles tensor. This tensor has a shape of [batch_size, num_triangles, 3, 3], where the last dimension represents the coordinates (x, y, z) of the three vertices of each triangle.

   x = self.triangles[:, :, :, 0]  # x-coordinates of the vertices
   y = self.triangles[:, :, :, 1]  # y-coordinates of the vertices
   z = self.triangles[:, :, :, 2]  # z-coordinates of the vertices

2. Volume Calculation: The volume of the mesh is computed using the determinant of a matrix that represents the signed volume of the tetrahedron formed by the origin and each triangle. For each triangle, the formula is derived from the volume of a tetrahedron: However, instead of directly computing the determinant, an equivalent expression is used to sum over all triangles. This expression leverages the fact that the sum of the determinants can be written in a simplified form, avoiding explicit determinant computation.

   volume = (
   -x[:, :, 2] * y[:, :, 1] * z[:, :, 0] +
   x[:, :, 1] * y[:, :, 2] * z[:, :, 0] +
   x[:, :, 2] * y[:, :, 0] * z[:, :, 1] -
   x[:, :, 0] * y[:, :, 2] * z[:, :, 1] -
   x[:, :, 1] * y[:, :, 0] * z[:, :, 2] +
   x[:, :, 0] * y[:, :, 1] * z[:, :, 2]
   ).sum(dim=1).abs() / 6.0

   These terms sum up the signed volumes of the tetrahedrons for each triangle in the mesh. The sum(dim=1) operation sums the volumes across all triangles, and abs() ensures the volume is positive. Finally, dividing by 6 normalizes the result.

But I still fail to understand how it would explain SMPL-X volume to be so much smaller, unless there is something fishy about the indexing.

[MarilynKeller]

I see, good use of chat gpt, that makes it clearer.

I looked at your current fork and I don't see you take into account the model nature in the body measurements computation: The BodyMeasurements object is instanciated through this function here. This function still builds a SMPL mesh from the beta and theta you provide to make the volume computation. If you are working on this branch, this is an issue for sure.

[MarilynKeller]

So when the measurements are computed given SMPL / SMPLX data (here), there should be an argument to specify if these data are smpl or smplx parameters, so that the BodyMeasurement object can be instanciated with the corresponding model instead of 'smpl' as it is now (here).

[davidpagnon]

Definitely, I was focused on the marker generation and looked over the body measurements. Will do now!

[davidpagnon]

Okay, that's much better now that I changed the measurement vertices and faces to SMPLX model type!

Height is correct to the centimeter, weight is still not perfect but we are down to a 2-3 kg difference. Considering how approximately this value is calculated anyway, I don't think it will make much of a difference to AddBiomechanics results.

[MarilynKeller]

That sounds great, thanks for the contribution David!

[davidpagnon]

You're welcome! I needed it anyway 😁 (and this repo made my life much easier than if I had to code it myself from scratch)