Run align_thuman.py, by report an error

[JianShu0721]

Hi, here is a problem when I'm running align_thuman.py.

return _VF.einsum(equation, operands) # type: ignore[attr-defined] RuntimeError: einsum(): operands do not broadcast with remapped shapes [original->remapped]: [1, 45]->[1, 1, 45] [12, 45]->[1, 45, 12]

I'm trying to use THuman2.0 to train the model, when preparing the dataset following the words in README.md:⚠️ Note that the scans in THuman2.0 are in various scales. We rescale them to -1~1 based on the SMPL-X models. You can find the rescaling script here

My environment: Ubuntu 20.04.6 Python 3.8 Pytorch 1.11.0+cu113

[azuxmioy]

Hi, thanks for your question and for reporting the spam🙂

We also have a newer version here in our SiTH repo.

But your issue seems to be using the wrong model to load the SMPL-X params.

Are you trying to process THuman2.1? Actually, 2.0 and 2.1 used different models

2.0 used the male model with PCA of hand with 12 components 2.1 used the neutral model without PCA (45 joints).

You need to correct these arguments at the beginning like here

Let me know if this solves your problem. Thanks!

[JianShu0721]

Hi, thanks for your question and for reporting the spam🙂

We also have a newer version here in our SiTH repo.

But your issue seems to be using the wrong model to load the SMPL-X params.

Are you trying to process THuman2.1? Actually, 2.0 and 2.1 used different models

2.0 used the male model with PCA of hand with 12 components 2.1 used the neutral model without PCA (45 joints).

You need to correct these arguments at the beginning like here

Let me know if this solves your problem. Thanks!

Hi, many thanks for your help and it works!

However, when I'm trying to use generate_dataset.py to make THuman.h5 file for training, it seems that this scripts cannot detect the materials. The problem is:

Traceback (most recent call last): File "/home/3Dhuman/LLEVH/editable-humans/generate_dataset.py", line 250, in main(parser.parse_args()) File "/home/3Dhuman/LLEVH/editable-humans/generate_dataset.py", line 132, in main rgb1, nrm1, d1 = closest_tex(V.cuda(), F.cuda(), File "/home/3Dhuman/LLEVH/editable-humans/lib/ops/mesh/closest_tex.py", line 59, in closest_tex rgb = sample_tex(hit_Tp, hit_TM, materials) File "/home/3Dhuman/LLEVH/editable-humans/lib/ops/mesh/sample_tex.py", line 28, in sample_tex assert(max_idx > -1 and "No materials detected! Check the material definiton on your mesh.") AssertionError

I use the aligned THuman data as input, the picture shows the structure of the dataset. Do you have any ideas? I'd appreciate it.

[azuxmioy]

Hello,

This might be a problem from the trimesh library since an update of its file IO.

I think I have already fixed this issue in the new version align_thuman.py in the SiTH repo.

A fast way to check this is to open your 0000.obj with Meshlab and see if it loads the material automatically.

If not, you can open the obj file as a txt file. The definition of the material file is located in the very beginning.

[JianShu0721]

Hello,

This might be a problem from the trimesh library since an update of its file IO.

I think I have already fixed this issue in the new version align_thuman.py in the SiTH repo.

A fast way to check this is to open your 0000.obj with Meshlab and see if it loads the material automatically.

If not, you can open the obj file as a txt file. The definition of the material file is located in the very beginning.

Hi, meshlab could load the material automatically. When I open the .obj file as tet, it looks like this:

The left is from the aligned THuman dataset while the right one is from Costomhuman dataset Also I think the .mti file is also correctly related too the material:

Is this a problem on the dataset? I use the new version align_Thuman to process data.

[JianShu0721]

Finished, it seems that in the file generate_dataset.py, there is a strange logic error. When trying to loading xxxx.obj file in the THuman dataset, the filename is still xxxx_smplx.obj. However this not happen on CustomHuman dataset.

[azuxmioy]

Thanks for the clarification😀.

Yes, it seems that the filename parsing in this line doesn't match the THuman file format.

Originally this repo was intended for the CustomHuman dataset, and I didn't notice this mistake when using THuman😅