Camera initialization is NaN loss value

[JackieXuu]

When I run this code in a folder with many some images, the code will stop sometimes. The error is

Found Trained Model: vposer_v1_0/snapshots/TR00_E096.pt
NaN loss value, stopping!
Camera initialization done after 2.0577
Traceback (most recent call last):
  File "smplifyx/main.py", line 276, in <module>
    main(**args)
  File "smplifyx/main.py", line 266, in main
    **args)
  File "/home/smplify-x/smplifyx/fit_single_frame.py", line 393, in fit_single_frame
    cam_init_loss_val))
TypeError: unsupported format string passed to NoneType.__format__

I think this error is about camera initialization loss. How to solve this problem? The second time I run this code for the same error image, it runs well.

[vchoutas]

Do you have some sample and instructions to re-produce this?

[Anirudh257]

@vchoutas I am also getting the same error on this image:

To reproduce, you can follow the following steps: 1) I put this image in a data_folder/images and the corresponding openpose keypoints in data_folder/keypoints directory.

2) I run

python smplifyx/main.py --config cfg_files/fit_smplx.yaml --data_folder DATA_FOLDER --output_folder OUTPUT_FOLDER --visualize="True/False" --model_folder MODEL_FOLDER --vposer_ckpt VPOSER_FOLDER --part_segm_fn smplx_parts_segm.pkl

where DATA_FOLDER is my data_folder containing this image.

3) I notice that the camera initialization that is being done in https://github.com/vchoutas/smplify-x/blob/e34b20c09f4de38099481e0d25b4f4536db5fc88/smplifyx/fit_single_frame.py#L270 has a nan value.

tensor([[0., 0., nan]], device='cuda:0')

I suspect that this issue is due to openpose and not smpl-x as SMPL-X is working properly on similar images.

[vchoutas]

@Anirudh257 For this example I am guessing it crashes because there is no left hip/shoulder for the person on the right. I could add a warning in those cases and ignore to prevent NaNs in fitting.

[Anirudh257]

@vchoutas That may be true. Another issue is that the person not performing the action [ guy] is getting detected with higher confidence as opposed to the performer [ girl ] causing the issue.