search

facebookresearch / tava

Code for the paper "TAVA Template-free Animatable Volumetric Actors".
BSD 3-Clause "New" or "Revised" License
202 stars 19 forks source link

Strange results from ZJU-Mocap 315 sequence #12

Open shanemankiw opened 1 year ago

shanemankiw commented 1 year ago

Hi, thank you for sharing your work!

I tried to train your model on ZJU-Mocap 315 sequence following your configuration, the only difference is that I used only one RTX 3090. The training took about 2 days yet the evaluation result seems strange.

This is val_ind result: 5253_315_2012_1

This is ood result: 0876_315_51_11

The metrics seem to be acceptable though. This is ind metrics log: step=10000, test_render_every=5253, psnr=27.6131, ssim=0.9523 step=20000, test_render_every=5253, psnr=28.0429, ssim=0.9556 step=30000, test_render_every=5253, psnr=28.2928, ssim=0.9566 step=40000, test_render_every=5253, psnr=27.9075, ssim=0.9584 step=50000, test_render_every=5253, psnr=28.7775, ssim=0.9605 step=60000, test_render_every=5253, psnr=28.2880, ssim=0.9582 step=70000, test_render_every=5253, psnr=28.5239, ssim=0.9606 step=80000, test_render_every=5253, psnr=28.9092, ssim=0.9616 step=90000, test_render_every=5253, psnr=28.6294, ssim=0.9594 step=100000, test_render_every=5253, psnr=28.6554, ssim=0.9586 step=110000, test_render_every=5253, psnr=28.7274, ssim=0.9584 step=120000, test_render_every=5253, psnr=28.6370, ssim=0.9589 step=130000, test_render_every=5253, psnr=28.4819, ssim=0.9584 step=140000, test_render_every=5253, psnr=28.7621, ssim=0.9583 step=150000, test_render_every=5253, psnr=28.6087, ssim=0.9575 step=160000, test_render_every=5253, psnr=28.4336, ssim=0.9555 step=170000, test_render_every=5253, psnr=28.6071, ssim=0.9572 step=180000, test_render_every=5253, psnr=28.5575, ssim=0.9559 step=190000, test_render_every=5253, psnr=28.5406, ssim=0.9569 step=200000, test_render_every=5253, psnr=28.5062, ssim=0.9558 step=210000, test_render_every=5253, psnr=28.2677, ssim=0.9553 step=220000, test_render_every=5253, psnr=28.3719, ssim=0.9560 step=230000, test_render_every=5253, psnr=28.3678, ssim=0.9565 step=240000, test_render_every=5253, psnr=28.3076, ssim=0.9561 step=250000, test_render_every=5253, psnr=28.3422, ssim=0.9564 step=260000, test_render_every=5253, psnr=28.3457, ssim=0.9562 step=270000, test_render_every=5253, psnr=28.2343, ssim=0.9560 step=280000, test_render_every=5253, psnr=28.2339, ssim=0.9557 step=290000, test_render_every=5253, psnr=28.1508, ssim=0.9561 step=300000, test_render_every=5253, psnr=28.2380, ssim=0.9562

This is ood metrics log: step=10000, test_render_every=876, psnr=24.9693, ssim=0.9287 step=20000, test_render_every=876, psnr=25.3594, ssim=0.9310 step=30000, test_render_every=876, psnr=25.3456, ssim=0.9363 step=40000, test_render_every=876, psnr=25.7967, ssim=0.9402 step=50000, test_render_every=876, psnr=25.8560, ssim=0.9385 step=60000, test_render_every=876, psnr=25.7005, ssim=0.9362 step=70000, test_render_every=876, psnr=25.9159, ssim=0.9374 step=80000, test_render_every=876, psnr=25.9287, ssim=0.9380 step=90000, test_render_every=876, psnr=25.9482, ssim=0.9385 step=100000, test_render_every=876, psnr=25.7621, ssim=0.9363 step=110000, test_render_every=876, psnr=25.6561, ssim=0.9364 step=120000, test_render_every=876, psnr=25.6878, ssim=0.9379 step=130000, test_render_every=876, psnr=25.6890, ssim=0.9383 step=140000, test_render_every=876, psnr=25.8649, ssim=0.9381 step=150000, test_render_every=876, psnr=25.8306, ssim=0.9383 step=160000, test_render_every=876, psnr=25.7063, ssim=0.9366 step=170000, test_render_every=876, psnr=25.5456, ssim=0.9353 step=180000, test_render_every=876, psnr=25.5683, ssim=0.9353 step=190000, test_render_every=876, psnr=25.5363, ssim=0.9356 step=200000, test_render_every=876, psnr=25.5620, ssim=0.9333 step=210000, test_render_every=876, psnr=25.4459, ssim=0.9329 step=220000, test_render_every=876, psnr=25.3567, ssim=0.9333 step=230000, test_render_every=876, psnr=25.3890, ssim=0.9334 step=240000, test_render_every=876, psnr=25.4219, ssim=0.9330 step=250000, test_render_every=876, psnr=25.3964, ssim=0.9336 step=260000, test_render_every=876, psnr=25.3680, ssim=0.9336 step=270000, test_render_every=876, psnr=25.3631, ssim=0.9335 step=280000, test_render_every=876, psnr=25.3580, ssim=0.9333 step=290000, test_render_every=876, psnr=25.3318, ssim=0.9334 step=300000, test_render_every=876, psnr=25.3467, ssim=0.9337

liruilong940607 commented 1 year ago

Hey sorry for the late reply. I didn't keep track of the issues here.

This is very odd. We did verified the code can reproduce the results in the paper. Have you already figured out something? Maybe a data setup issue?