ubisoft / ubisoft-laforge-ZeroEGGS

All about ZeroEggs
Other
360 stars 59 forks source link

How can I control PCA components in the code? #15

Open tieincred opened 1 year ago

tieincred commented 1 year ago

Hello, @saeedakbar Thank you for your amazing work, I would like to know how can I control PCA components in the code. as the paper mentions that we can control the root velocity, hand trajectory, and styles, how can I manipulate it in the code to visualize various results? I tried checking out the inference file but the couldn't see anything obvious enough. Can you please guide me with this?

Miru302 commented 1 year ago

If I understood correctly, you will need to compute style embedding arrays using style encoder and use Principal Component Analysis on that data to convert each style embedding into a point in PCA space. Then according to paper PCA axis can be used to control a feature in animation like root velocity etc. Although in my tests PCA plot was scattered with no human definable feature correlation with an axis.

Edit: btw, the problem I encountered, it was my error. But while I could control hip sway and hand gesticulation to some extent, it's not very precise control method. #46

tieincred-pixis commented 1 year ago

@Miru302 That sounds alright, Thanks for the insight