sebastianstarke / AI4Animation

Bringing Characters to Life with Computer Brains in Unity
7.4k stars 1.03k forks source link

Questions about "DeepPhase: Periodic Autoencoders for learning motion phase manifolds" #86

Open wengn opened 1 year ago

wengn commented 1 year ago

Hi, Sebastian

How are you? We are following closely with your research work on applying Deep Learning onto character animation, and I want to say they are great work! We are reading your Siggraph 2022 paper "DeepPhase: Periodic Autoencoders for Learning Motion Phase Manifolds" and trying to reproduce the work, but got stuck on some questions. I am wondering if you could help me with these detailed questions.

  1. What's the kernel-size of the convolultional layer?
  2. What method did you use to initialize the weights?
  3. What are the validation/test loss you achieved after you finished training?
  4. If I change the kernal size, there are quite a few occations that loss became nan,do you know what could be the reason for this?
  5. In the paper, does every channel connect to a unique fully connected layer? What's the activation function of the fully connected layer?
  6. Does the FFT layer has weights to learn as well?
  7. The sampling time for a time window is 2 second, correct? Also the T in "f" in formula (3) is also 2 seconds, right?

We used your dataset from paper "Neural State Machine for Character-Scene Interactions",but the lowest loss we could get is 0.2. We think it is too high and don't find a way to reduce it. Can you shed some light on this?

Avoid 18863(5.24min) Carry 53094(14.75min) Crouch 7659 (2.13min) Door 58479(16.24min) Jump 4511 (1.25min) Loco 59859(16.63min) Sit 199472(55.41min) total: 401937 (111min)

Thanks a lot!