search

c-he / NeMF

[NeurIPS 2022] Official implementation of "NeMF: Neural Motion Fields for Kinematic Animation"
MIT License
155 stars 9 forks source link

should varaible "rotmat_recon" in decode function match to 'global_xform' ? #15

Open ChangjianLi opened 1 year ago

ChangjianLi commented 1 year ago

hi, I have a little question about the generative model. I looked throught the code, in decode function, the varaible "rotmat_recon" is transformed from "local_motion". And it is transformed into varaible "local_rotmat" using function self.fk.global_to_local() for calculating "pos_recon" image so can it be regarded as the reconstructed "global_xform"? The "rotmat_recon" is further assigned as output['rotmat']. In backward() function , it is used to calculate the rotmat_recon_loss by comparing with self.input_data['rotmat']. But does the gt varaible self.input_data['rotmat'] should be gt "local_rotmat" instead of gt "global_xform"? Because in data processing the varaible "rotmat" is calculated straight from smpl data image

c-he commented 1 year ago

Hi Changjian, sorry for the confusion caused by my bad coding style. rotmat in data processing are local rotation matrices, but they are actually not used in the training code. In generative.py, rotmat are actually rotation matrices converted from global transformation, as shown in:

https://github.com/c-he/NeMF/blob/79918430970fd138ae730510459c8f34893a3f86/src/nemf/generative.py#L102

ChangjianLi commented 1 year ago

Haha ok thx, that makes it clear to me, thank you for the reply >_<