About quantitative evaluation

[mkstmyk]

Hi @DK-Jang, thank you for your great paper w/ the code! I'd like to reproduce the evaluation for my further research. While waiting for your release of quantitative evaluation code, I made a ST-GCN based content/style classifiers by modifying the discriminator of Diverse Motion Stylization which is a previous work from your group. The classifier showed enough accuracies (around 0.9) similar to yours reported in the paper on real motions (Xia's dataset). However, they showed different CRA (0.73) and SRA (0.14) on generated data stylized by MotionPuzzle. I'm afraid that I'm doing something wrong with my classifiers or the way of using MotionPuzzle on Xia's dataset. I appreciate if you release the quantitative evaluation codes, but I know you are too busy to work on it. Before finalizing the evaluation code, could you tell me the detail of the pre-trained classifier used in the paper?

[zhuzhuruk]

I doubt your experimental results are correct. I run the demo multiple times with random style from XIA dataset and random content from CMU dataset. However it can be noticed that the style appeared in the generated motion is quite weird and just look like simply copying the "something" (not the style understood by human) of the style motion to the content motion. Thus the style is hardly recognized.

[mkstmyk]

@zhuzhuruk @DK-Jang Thank you for your reply! (and sorry for my late response) I also noticed such weird cases, which is rare though. From my observation, in some cases, content of style motion is reflected on the stylized motion. For example, when I stylize neutral_punching by referencing depressed_kicking as style motion, the generated motion seems to be a bit freezing, neither punching nor kicking are properly included but cancel each other out. Is this a known issue?