data process about 2019_NSM

[miaoYuanyuan]

I found the 2018_MANN code support data processing from raw bvh data. but the 2019_NSM only support Export from already assert files. I guess the reason that 2019_NSM not supprt Export from raw BVH data is it need some mannual label step which must be pre-processed before . is it that ?

[sebastianstarke]

Yea that is correct. I know this is not very ideal, but unfortunately working with geometry and scene data required us some more complex data processing, for which we weren't able to write a fully-automatic pipeline. Instead, we had multiple smaller scripts that helped us assigning labels, processing geometry or fitting object trajectories, so the amount of manual work was still not too high. Please let me know if any questions!

[miaoYuanyuan]

Thank you ! I have some problems.

1. the phase ,action,contact labels all need processed by manual. but the data argument can be automatic. Is there any data argument process in the project? such as : random choose chairs, reset the contact point ,full body Ik. or generate cylinder Sensor and environment/object Voxel .....
2. for one motion assert file, such as sit motin, the order of the random chairs is already fixed in it or random when Export?
3. In the Phase Module, I found the RegularPhase is 2pi when the left foot on the ground(left foot's velocity reduce to zero). but When I convert assert file to json file , and read the RegularPhase, I found it was 2pi when right foot on the ground. I want to know which is correct? is it correspond as PFNN ?

[zll961020]

@sebastianstarke I also have some problem about the Data Augmentation, is that done offline or calculate for each frame when labeling? By the way, where is the corresponding part of the code in the Unity project? Hope for your reply, thank you very much!

[zll961020]

@sebastianstarke Another question about the Object/Environment Fitting in Section 5.2 of the paper "Neural State Machine for Character-Scene Interactions" , the Step1 and Step2 are processed manually or there are some scripts that can process automatically?

[Hlxwk]

@sebastianstarke I also have some problem about the Data Augmentation, is that done offline or calculate for each frame when labeling? By the way, where is the corresponding part of the code in the Unity project? Hope for your reply, thank you very much!

Data Augmentation is done offline!

[sebastianstarke]

Yes the data augmentation is done offline and we had a number of small scripts to process the data. Unfortunately I do not have those small scripts anymore as those were quite hacked and required custom parameters for different clips, and only the final processed motion data in the asset files that contains all the annotations is uploaded here.