tijiang13
commented
5 months ago Hi bayun,
I just reran the demo and here is what I got after a few epochs:
Usually even if the pose is bad it would only impact the reconstruction quality to some extent. However, the result you showed appears to be complete divergence.
For troubleshooting, you can try overwriting the poses_optimized.npz and train.npz with data.zip and run bash ./bash/run-neuman-demo.sh again (remember to delete /home/wh/project/InstantAvatar/outputs/neuman/baseline/seattle to avoid confusion with previous runs)
Let me know if the problem persist.
Best, Tianjian
bayunCC
Thanks for your great job.
I set epoch to 400 didn't improve the result.
And I also got bad result after running train.py like this:
What I got after processing data:
I can't get good body model in using
bash run-neuman-demo.sh. I got obscure images in InstantAvatar/outputs/neuman/baseline/seattle/animation/progression when performing fit.py like this images:log: bash ./bash/run-neuman-demo.sh QOpenGLContext::swapBuffers() called with non-exposed window, behavior is undefined QOpenGLContext::swapBuffers() called with non-exposed window, behavior is undefined Duration: 1.32s @ 120.05 FPS Global seed set to 42 Switch to /home/wh/project/InstantAvatar/outputs/neuman/baseline/seattle [train] No optimized smpl found. [val] No optimized smpl found. [test] No optimized smpl found. [2024-04-19 10:04:40,355][torch.distributed.nn.jit.instantiator][INFO] - Created a temporary directory at /tmp/tmp3iywwkpd [2024-04-19 10:04:40,355][torch.distributed.nn.jit.instantiator][INFO] - Writing /tmp/tmp3iywwkpd/_remote_module_non_scriptable.py Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off] Loading model from: /home/wh/project/InstantAvatar/third_parties/lpips/weights/v0.1/vgg.pth GPU available: True, used: True TPU available: False, using: 0 TPU cores IPU available: False, using: 0 IPUs Saving configs. LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
| Name | Type | Params 0 | net_coarse | NeRFNGPNet | 13.0 M 1 | SMPL_param | SMPLParamEmbedding | 3.1 K 2 | loss_fn | NGPLoss | 14.7 M 3 | renderer | Raymarcher | 0
13.0 M Trainable params 14.7 M Non-trainable params 27.8 M Total params 111.022 Total estimated model params size (MB) /home/wh/anaconda3/envs/ins/lib/python3.8/site-packages/pytorch_lightning/trainer/data_loading.py:412: UserWarning: The number of training samples (41) is smaller than the logging interval Trainer(log_every_n_steps=50). Set a lower value for log_every_n_steps if you want to see logs for the training epoch. rank_zero_warn( Epoch 399: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42/42 [00:09<00:00, 4.47it/s, v_num=0] Save optimized pose to /home/wh/project/InstantAvatar/data/custom/seattle/poses/train.npz
Global seed set to 42 Switch to /home/wh/project/InstantAvatar/outputs/neuman/baseline/seattle [train] Loading from /home/wh/project/InstantAvatar/data/custom/seattle/poses/train.npz [val] No optimized smpl found. [test] No optimized smpl found. [2024-04-19 10:34:55,128][torch.distributed.nn.jit.instantiator][INFO] - Created a temporary directory at /tmp/tmpps0wff_b [2024-04-19 10:34:55,128][torch.distributed.nn.jit.instantiator][INFO] - Writing /tmp/tmpps0wff_b/_remote_module_non_scriptable.py Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off] Loading model from: /home/wh/project/InstantAvatar/third_parties/lpips/weights/v0.1/vgg.pth GPU available: True, used: True TPU available: False, using: 0 TPU cores IPU available: False, using: 0 IPUs Saving configs. LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
| Name | Type | Params 0 | net_coarse | NeRFNGPNet | 13.0 M 1 | loss_fn | NGPLoss | 14.7 M 2 | renderer | Raymarcher | 0
13.0 M Trainable params 14.7 M Non-trainable params 27.8 M Total params 111.009 Total estimated model params size (MB) /home/wh/anaconda3/envs/ins/lib/python3.8/site-packages/pytorch_lightning/callbacks/model_checkpoint.py:631: UserWarning: Checkpoint directory checkpoints/ exists and is not empty. rank_zero_warn(f"Checkpoint directory {dirpath} exists and is not empty.") /home/wh/anaconda3/envs/ins/lib/python3.8/site-packages/pytorch_lightning/trainer/data_loading.py:412: UserWarning: The number of training samples (41) is smaller than the logging interval Trainer(log_every_n_steps=50). Set a lower value for log_every_n_steps if you want to see logs for the training epoch. rank_zero_warn( Epoch 399: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 42/42 [00:24<00:00, 1.68it/s, v_num=0] Global seed set to 42
Switch to /home/wh/project/InstantAvatar/outputs/neuman/baseline/seattle [train] Loading from /home/wh/project/InstantAvatar/data/custom/seattle/poses/train.npz [val] No optimized smpl found. [test] No optimized smpl found. [2024-04-19 11:49:57,490][torch.distributed.nn.jit.instantiator][INFO] - Created a temporary directory at /tmp/tmp3n7mncvq [2024-04-19 11:49:57,491][torch.distributed.nn.jit.instantiator][INFO] - Writing /tmp/tmp3n7mncvq/_remote_module_non_scriptable.py Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off] Loading model from: /home/wh/project/InstantAvatar/third_parties/lpips/weights/v0.1/vgg.pth Resume from checkpoints/last.ckpt 60it [03:42, 3.71s/it] Global seed set to 42 Switch to /home/wh/project/InstantAvatar/outputs/neuman/baseline/seattle [train] Loading from /home/wh/project/InstantAvatar/data/custom/seattle/poses/train.npz [val] No optimized smpl found. [test] No optimized smpl found. [2024-04-19 11:53:49,076][torch.distributed.nn.jit.instantiator][INFO] - Created a temporary directory at /tmp/tmp8jecsoss [2024-04-19 11:53:49,076][torch.distributed.nn.jit.instantiator][INFO] - Writing /tmp/tmp8jecsoss/_remote_module_non_scriptable.py Setting up [LPIPS] perceptual loss: trunk [vgg], v[0.1], spatial [off] Loading model from: /home/wh/project/InstantAvatar/third_parties/lpips/weights/v0.1/vgg.pth Resume from checkpoints/last.ckpt 320it [11:41, 2.19s/it]