Geometry crash when training with custom animation in SMPL format

[Masoudvahid]

Hello,

I have a set of custom animations formatted as smpl_params.npz, and I am trying to train the model using these animations. However, when I attempt to train on my own animation data, the training crashes with geometry crashed error.

Strangely, if I use the default animations provided with the model, everything works correctly, and the training proceeds without any errors. It is worth noting that I am also training on my own dataset

Any help or insight into why this might be happening would be greatly appreciated!

Thank you!

[shad0wta9]

I'm not sure about the problem given the limited information. What is "It is worth noting that I am also training on my own dataset"? The training data consists of the captured images, the camera calibration (calibration_full.json) and pose sequence (smpl_params.npz). Which default components are you using? I think more specific description will be helpful.

[Masoudvahid]

Thank you very much for your quick response

By my own dataset I mean I have my own images, masks, calibration_full.json and smpl_params.npz. The training runs without any error when I use the smpl_params.npz in AvatarReX Dataset. But when I use my own pose sequence file, I get the following error:

iter=    0, deformation_loss=0.024882, perc_loss=0.016020, img_loss=0.469778, reg_loss=0.278869, lr=0.00000, time=1847.8 ms, rem=51.33 h
Traceback (most recent call last):
  File "/Avatar/MeshAvatar/train.py", line 468, in <module>
    geometry, mat = optimize_mesh(glctx, geometry, mat, lgt, dataset_train, dataset_validate,
  File "/Avatar/MeshAvatar/train.py", line 315, in optimize_mesh
    losses = trainer(target, it + FLAGS.start_iter, denoiser)
  File "/anaconda3/envs/meshavatar/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/anaconda3/envs/meshavatar/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
    return forward_call(*args, **kwargs)
  File "/Avatar/MeshAvatar/train.py", line 179, in forward
    return self.geometry.tick(glctx, target, self.light, self.material, self.image_loss_fn, it, denoiser, self.perc_loss_fn)
  File "/Avatar/MeshAvatar/geometry/dmtet.py", line 333, in tick
    buffers, cano_pos, motion_feat = self.render(glctx, target, lgt, opt_material, denoiser, shadow_ramp, training=True)
  File "/Avatar/MeshAvatar/geometry/dmtet.py", line 299, in render
    assert False, "The geometry crashed (no triangles)! Please restart the program!"
AssertionError: The geometry crashed (no triangles)! Please restart the program!

I am attaching my own pose file smpl_params.zip

[shad0wta9]

The crash error usually occurs when the camera calibration and smpl registration doesn't match. You can first check the image img_dmtet_pass1_000000.png in the out/<exp_name> folder. There should be an orb-like geometry at the same position as the human body in the input image.

[shad0wta9]

It seems very weird that there is no error when you only replace the smpl registration, since the calibration in your custom dataset shouldn't match the smpl poses in AvatarReX dataset.

[Masoudvahid]

It seems very weird that there is no error when you only replace the smpl registration, since the calibration in your custom dataset shouldn't match the smpl poses in AvatarReX dataset.

Could you please share guideline or repo where I can reconstruct the smpl poses? I have doubt that I have done them correctly.

[Masoudvahid]

The crash error usually occurs when the camera calibration and smpl registration doesn't match. You can first check the image img_dmtet_pass1_000000.png in the out/<exp_name> folder. There should be an orb-like geometry at the same position as the human body in the input image.

I have orb-like shape, but it is located in lower part of the output image Custom dataset: avatarex:

It would be appriciated if you could inform me about the correct procedure of creating the camera calibration and poses correctly. Thank you!

[shad0wta9]

It seems very weird that there is no error when you only replace the smpl registration, since the calibration in your custom dataset shouldn't match the smpl poses in AvatarReX dataset.

Could you please share guideline or repo where I can reconstruct the smpl poses? I have doubt that I have done them correctly.

check EasyMocap.