search

pbaylies / stylegan-encoder

StyleGAN Encoder - converts real images to latent space
Other
740 stars 182 forks source link

Finding latent representation in other domain #17

Closed JanBoskamp closed 5 years ago

JanBoskamp commented 5 years ago

First of all, thanks Pbaylies for sharing this great repo! Hopefully you can help me with the following.

How can I find a latent representation of myself, in an portrait-art stylegan model? Right now, I can find a good latent representation of myself, but the generated image is not in portrait art style. This generated image is in the style of the original FFHQ stylegan model, just similar as the image of myself. How can I achieve what e.g. aiportraits.com has, showing the most similar representation of a person in a portrait-art stylegan model? I have tried to play with parameters and cross-over latent codes, but still without succes.

pbaylies commented 5 years ago

Hi @JanBoskamp -- you'd want to use a StyleGAN model that's been trained on portraits, take a look at the notebook in this repo, by @ak9250

For the encoder parameters, you'll need to specify the model_url and the model_res (512 in this case) as well. Here's an example of what you can expect.

JanBoskamp commented 5 years ago

Hi @pbaylies , thanks for your answer, but let me clarify what I mean. With the Stylegan model I have trained, I can create portrait art (similar to @ak9250) and that looks like real art. Very cool. But when I try to find a latent representation in this art domain for an FFHQ image, or an image of myself, the generated image doesn't look like art. Although I am only using the art-stylegan model and not the well-known FFHQ stylegan model to find the representations.

I have attached two example art-stylegan results, and two found latent representations and their images. Any idea how I can force the encoding process to stay more in the art domain? In other words, how can I generate art-stylegan images, with similar facial features as the images of faces I use as input. The example of Elon Musk is the goal I try to achieve with my own model.

Thanks again!

image image image

pbaylies commented 5 years ago

Oh hey, nice model, I'd love to try that out! What you should try is raising the L1 penalty (the --use_l1_penalty flag), that will pull it more towards the center of the distribution, i.e., the art domain. Try raising it to 2 or 4, probably 20 would be too high, but somewhere in there.

portnoyv commented 5 years ago

Looks great, Can you share the pre trained wiki-art network? Also, did you train resnet/effnet on wikiart data?

portnoyv commented 4 years ago

@JanBoskamp did you succeed?