EvgenyKashin / stylegan2-distillation

https://arxiv.org/abs/2003.03581
Other
1.01k stars 105 forks source link

Learning directions #20

Closed aomv closed 3 years ago

aomv commented 3 years ago

Hi,

I’ve followed your notebook where you suggest to train directions based on the 1x512 latents and then tile the resulting directions to 18x512. Unfortunately it doesn’t really work. I have 300k images annotated with the Microsoft API and I would love to contribute more directions to this repo but none of the directions I learn work as well as the ones provided. Could you clarify how exactly those directions where trained?

I’ve tried training in either the 1x512 and 18x512 but even a simple gender direction learning based on Puzer’s original notebook won’t work well.

On his notebook he trained on the mapped latents 18x512 and when he applied then he use a [:8] portion of the latent only. Your notebook suggests training on the 1x512 and applying a tile to save the learned direction as 18x512. Unfortunately that process doesn’t work.

Would you mind updating your notebook with direction learning instructions that can be reproduced ? It would be great if you could clarify exactly how the provided directions were trained and how exactly to apply them (if use use a portion only of the learned direction as Puzer or if you use the whole direction vector when adding to the latent)

Thank you

aomv commented 3 years ago

This is what 120k images annotated with the Microsoft API as male/female will get you when you try to train them on the notebook provide on this repo. As you can see it doesn't work. It makes me wonder if the directions available on this repo were trained with some other way. Maybe with the more recent unsupervised gan exploration techniques. Don't waste your time trying to apply the recommended logistic regression suggested in this repo.

Screen Shot 2020-12-13 at 7 49 16 PM
0x454447415244 commented 3 years ago

Not sure about the notebook, but you can use the provided directions to generate images for training (e.g., 50k).

EvgenyKashin commented 3 years ago

Hi, thanks for posting you experiments results!

About provided vectors - I got them from robertluxemburg for this repo (as written in readme). In the paper we used internal Yandex's API (because of that we couldn't share our vectors) for synthetics classification and we used provided notebook for training a logreg (as you mentioned - it's basically as in Puzer's notebook).

We applied vectors only on the first 8th components, as you could find in the code.