search

taconite / PTF

[CVPR'21] Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration
https://taconite.github.io/PTF/website/PTF.html
MIT License
84 stars 8 forks source link

Problems of SMPL/SMPL+D Registration #7

Closed Bingoang closed 2 years ago

Bingoang commented 2 years ago

@taconite Hi, taconite When I run the command python smpl_registration/fit_SMPLD_PTFs.py --num-joints 24 --use-parts --init-pose configs/cape/${config}.yaml, the output logs are as follows:

(PTF) ang@ang-All-Series-Invalid-entry-length-16-Fixed-up-to-11:/media/ang/PTF-main$ python smpl_registration/fit_SMPLD_PTFs.py --num-joints 24 --use-parts --init-pose configs/cape/ptf_decoder-width-256-128_ptfs-width-256-128_full-smpl_ce-ce_w-skin-1e-1_conv-encoder_hidden-256_plane64x3_softmax_npts-5000_CAPE-release-with-aug-trans_1gpus.yaml Warning: unable to import datasets/nusc: No module named 'nuscenes' Traceback (most recent call last): File "/media/ang/PTF-main/kaolin/kaolin/datasets/init.py", line 11, in from .nusc import NuscDetection File "/media/ang/PTF-main/kaolin/kaolin/datasets/nusc.py", line 21, in from nuscenes.utils.geometry_utils import transform_matrix ModuleNotFoundError: No module named 'nuscenes' Warning: unable to import datasets/nusc: None Using BLB SMPL from the project: LearningRegistration /media/ang/PTF-main/im2mesh/config.py:19: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. cfg_special = yaml.load(f) /media/ang/PTF-main/im2mesh/config.py:30: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. cfg = yaml.load(f) 0%| | 0/331 [00:00<?, ?it/s](6890, 3, 10) /media/ang/PTF-main/smpl_registration/lib/smpl_layer.py:54: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /opt/conda/conda-bld/pytorch_1595629403081/work/torch/csrc/utils/tensor_numpy.cpp:141.) torch.Tensor(smpl_data['betas'].r).unsqueeze(0)) Optimizing SMPL global orientation | 0/30 [00:00<?, ?it/s] Iter: 29, s2m: 0.1590, m2s: 0.2704, betas: 0.0418, pose_pr: 0.0004, part: 0.5812: 100%|██████████████| 30/30 [05:54<00:00, 11.83s/it] Iter: 29, s2m: 0.1590, m2s: 0.2704, betas: 0.0418, pose_pr: 0.0004, part: 0.5812: 100%|██████████████| 30/30 [05:54<00:00, 12.03s/itOptimizing SMPL pose only | 0/30 [00:00<?, ?it/s] Iter: 29, s2m: 0.0200, m2s: 0.0064, betas: 0.0014, pose_pr: 0.0003, part: 0.0155: 100%|██████████████| 30/30 [06:05<00:00, 12.19s/it] Iter: 29, s2m: 0.0203, m2s: 0.0030, betas: 0.0002, pose_pr: 0.0003, part: 0.0078: 100%|██████████████| 30/30 [06:16<00:00, 12.54s/it] Optimised smpl pose s: 0.0030, betas: 0.0002, pose_pr: 0.0003, part: 0.0078: 100%|██████████████| 30/30 [06:16<00:00, 12.48s/it] Iter: 29, s2m: 0.0095, m2s: 0.0105, betas: 0.0007, pose_pr: 0.0008, part: 0.0287: 100%|██████████████| 30/30 [06:17<00:00, 12.59s/it] Iter: 29, s2m: 0.0131, m2s: 0.0035, betas: 0.0004, pose_pr: 0.0004, part: 0.0107: 100%|██████████████| 30/30 [06:19<00:00, 12.64s/it] Iter: 29, s2m: 0.0176, m2s: 0.0023, betas: 0.0006, pose_pr: 0.0003, part: 0.0071: 100%|██████████████| 30/30 [06:18<00:00, 12.63s/it] Optimised smpl pose and shape betas: 0.0006, pose_pr: 0.0003, part: 0.0071: 100%|██████████████| 30/30 [06:18<00:00, 12.87s/it] Lx100. Iter: 9, s2m: 0.0023, m2s: 0.0032, lap: 0.0007, offsets: 0.0044: 100%|████████████████████████| 10/10 [02:11<00:00, 13.18s/it] Lx100. Iter: 9, s2m: 0.0010, m2s: 0.0015, lap: 0.0004, offsets: 0.0038: 100%|████████████████████████| 10/10 [02:13<00:00, 13.35s/it] Lx100. Iter: 9, s2m: 0.0004, m2s: 0.0008, lap: 0.0002, offsets: 0.0039: 100%|████████████████████████| 10/10 [02:08<00:00, 12.87s/it] Lx100. Iter: 9, s2m: 0.0002, m2s: 0.0005, lap: 0.0001, offsets: 0.0041: 100%|████████████████████████| 10/10 [02:08<00:00, 12.90s/it] Lx100. Iter: 9, s2m: 0.0001, m2s: 0.0004, lap: 0.0001, offsets: 0.0042: 100%|████████████████████████| 10/10 [02:09<00:00, 12.94s/it] Inner distance for input shortlong_ATUsquat.000001: 0.01631784997880459 cm0%|████████████████████████| 10/10 [02:09<00:00, 12.87s/it] Outer distance for input shortlong_ATUsquat.000001: 0.017044199630618095 cm Inner distance for input shortlong_ATUsquat.000006: 0.013444711454212666 cm Outer distance for input shortlong_ATUsquat.000006: 0.014601950533688068 cm Inner distance for input shortlong_ATUsquat.000011: 0.015537642873823643 cm Outer distance for input shortlong_ATUsquat.000011: 0.0160372331738472 cm Inner distance for input shortlong_ATUsquat.000016: 0.014674867503345013 cm Outer distance for input shortlong_ATUsquat.000016: 0.015363145619630814 cm Inner distance for input shortlong_ATUsquat.000021: 0.013980861753225327 cm Outer distance for input shortlong_ATUsquat.000021: 0.014812660403549671 cm Inner distance for input shortlong_ATUsquat.000026: 0.014417361468076706 cm Outer distance for input shortlong_ATUsquat.000026: 0.014523924328386784 cm Inner distance for input shortlong_ATUsquat.000031: 0.014444430358707905 cm Outer distance for input shortlong_ATUsquat.000031: 0.015400375239551067 cm Inner distance for input shortlong_ATUsquat.000036: 0.014390168711543083 cm Outer distance for input shortlong_ATUsquat.000036: 0.014232808724045753 cm Inner distance for input shortlong_ATUsquat.000041: 0.01653471030294895 cm Outer distance for input shortlong_ATUsquat.000041: 0.017072072252631187 cm Inner distance for input shortlong_ATUsquat.000046: 0.01617429219186306 cm Outer distance for input shortlong_ATUsquat.000046: 0.015451897867023945 cm Inner distance for input shortlong_ATUsquat.000051: 0.01808827929198742 cm Outer distance for input shortlong_ATUsquat.000051: 0.017144061625003815 cm Inner distance for input shortlong_ATUsquat.000056: 0.016339648514986038 cm Outer distance for input shortlong_ATUsquat.000056: 0.016375551000237465 cm 0%|▎ | 1/331 [48:37<267:23:39, 2917.03s/it](6890, 3, 10) Optimizing SMPL global orientation | 0/30 [00:00<?, ?it/s]

Iter: 15, s2m: 0.8738, m2s: 1.0011, betas: 0.0114, pose_pr: 0.0014, part: 3.3756: 53%|███████▍ | 16/30 [04:05<03:34, 15.29s/it] .....

It seems that there're 2 problems: 1) No module named 'nuscenes': Is it matters? We do not use the nuscenes dataset after all. 2) PTF-main/smpl_registration/lib/smpl_layer.py:54: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. Although it warns, it can output the registration .ply files. I wonder if I can ignore the warning?

Thanks in advance, and look forward to your reply!

taconite commented 2 years ago

Hi, Bingoang

Sorry for the late reply, I was busy preparing our NeurIPS presentation and code release

1) This should not affect the code; it is some optional dependency of kaolin, which we never actually used. 2) You can ignore the warning since these underlying numpy arrays were never directly modified after creation.

best, Shaofei

Bingoang commented 2 years ago

@taconite Thanks very much for your reply, and looking forward to your new interesting works!