search

c-he / NeMF

[NeurIPS 2022] Official implementation of "NeMF: Neural Motion Fields for Kinematic Animation"
MIT License
155 stars 9 forks source link

Bug Appears While Trying to Create BVH Viz with Generative Model #20

Open archerguzen opened 10 months ago

archerguzen commented 10 months ago

Hello, I was trying to run the generative model with my custom BVH dataset. So I followed a similar approach while training the model and generating the test outputs. As you did in generative.py, I used BVH.save function to save the generated anim that is created with positions and rotations arrays. It does not give any errors if the step = 1. But while generating the 60 FPS animation with step = 0.5, it gives an error. The error is caused by the positions array.

While creating the positions array, if 'trans' key does not exist inside the reconstructed data, it creates a zero array with the shape of self.input_data['trans']:

https://github.com/c-he/NeMF/blob/79918430970fd138ae730510459c8f34893a3f86/src/nemf/generative.py#L451-L458

The reconstructed 60 FPS animation contains 2T amount of frames while the input data contains only T amount of frames. That is why the positions array also contains T amount of frames. Since the rotations and positions arrays do not have the matching frame count, the program gives an error.

I fixed this issue by simply passing (b_size, rotmat.shape[1], 3) as the shape parameters in the creation of the positions array. I did not check if I get any errors if the 'trans' key is available in reconstructed data.

c-he commented 10 months ago

Thanks for pointing it out! I will pin this issue for anyone who run into similar issues.