Running on ZJU dataset

[jamesyoung0623]

Hello, Thank you for sharing your work! I am trying to run your training code on the ZJU dataset, but here's my result.

If you have experimented with the ZJU dataset would you share your experiment? Thanks a lot!

[tijiang13]

Hi James,

Thank you for sharing the results. Although I haven't yet tested the model on the ZJU dataset, given that they have calibrated cameras and poses estimated from a multi-view setup this dataset should be somewhat easier to work with compared to the Neuman dataset.

Regarding the results, I remember that the ZJU dataset uses a modified version of the SMPL model, whereas we adhered to the standard SMPL definition. Did you convert the SMPL parameters accordingly? You can easily verify the SMPL parameters using the visualization script available here."

Best, Tianjian

[jamesyoung0623]

Thank you for pointing out the difference and the useful script. I have another question about anim_nerf_train.npz, anim_nerf_val.npz, and anim_nerf_test.npz. I notice that they have 114, 1, and 55 frames respectively. Are these frames randomly sampled from the origin 690 frames?

Also, what do the 'transl' parameters in poses.npz stand for? Thank you!

[tijiang13]

Hi James,

1. They are sampled in equal steps. The length varies as the original videos have different lengths.
2. it's the global translation of SMPL.

Best, Tianjian

[jamesyoung0623]

Hi Tianjian, Thank you for your explanation. This is my current visualization.

https://github.com/tijiang13/InstantAvatar/assets/55226860/47e85a61-c4ff-4015-a704-b5790a680487

It seems like the rotation of ZJU dataset is incompatible with yours. Would you know how to rotate the pose parameters? Thank you!

[tijiang13]

Hi James,

You can check this for the conversion.

Best, Tianjian