una-dinosauria / human-motion-prediction

Simple baselines and RNNs for predicting human motion in tensorflow. Presented at CVPR 17.
MIT License
406 stars 141 forks source link

A few question about data processing of human3.6m #15

Closed YongyiTang92 closed 7 years ago

YongyiTang92 commented 7 years ago

Hi Julieta, I am confus about why a skeleton has 99 dims presentation? And I find that the rotMat2euler function is quite different to the conventional formulas which I think correctly implement in https://github.com/xioTechnologies/Quaternion-MATLAB-Library/blob/master/rotMat2euler.m . Do you have any idea about this?

una-dinosauria commented 7 years ago

Hi @YongyiTang92,

I am confus about why a skeleton has 99 dims presentation?

We take the data as pre-processed by Jain et al in their CVPR'16 paper "Structural RNN" (that's why the link is from stanford.edu: http://www.cs.stanford.edu/people/ashesh/h3.6m.zip).

Unfortunately, I don't think the authors made their pre-processing code available, but I believe they simply converted the H3.6M dataset to exponential map, and are making sure that every joint is represented by 3 numbers -- some joints never move (eg, you can't move your knee sideways), so some dimensions end up being constants, which are removed when you get rid of the dimensions that have a very small standard deviation. This is what we do in our code.

I find that the rotMat2euler function is quite different to the conventional formulas which I think correctly implement in https://github.com/xioTechnologies/Quaternion-MATLAB-Library/blob/master/rotMat2euler.m . Do you have any idea about this?

I am not sure what you mean by this. Are you saying that our implementation has a bug? If so, please open an issue with code to reproduce the error that you are getting.

Our implementation is a python port of the function by Graham Taylor (https://github.com/asheshjain399/RNNexp/blob/srnn/structural_rnn/CRFProblems/H3.6m/mhmublv/Motion/RotMat2Euler.m#L1). I haven't spent much time looking at your implementation, but the one we use seems a bit more numerically stable (e.g. consider what would happen if the denominator is very close to zero in this line). If you want to dig deeper into this I recommend you to come up with a series of tests so that we can learn

Cheers,