Open citymonkeymao opened 6 years ago
Hi, it's interesting. what are related papers of your code. I wanna take a look.
I didn't see any paper describing this yet. However, you can look at this paper that proposed GAN. Here I use GAN to capture the distribution of gram matrices.
I think it's better to calculate correct prediction percent of D for fakes and reals.
The correct rates of discriminators fluctuate except the 4th style layer. the discriminator can always success on this layer. I guess this is because this layer is also used as the content layer.
So what the limitations of your idea here? Is the image size limited to 256 like it is in CycleGAN? Do we need a thousand image dataset and pretrained CycleGAN model for each artists' style?
Like the original style transfer, you can decide image size by using -image_size
option.The only limitation for this is your memory.
I'm not sure the lower limit of the images needed, the Shin style demonstrated there used 170 images.
Learn from multiple styles with GAN
When using hundreds of pictures as style images, a discriminator could be used to calculate the style loss. The discriminator takes gram matrix as input and was trained to tell whether the generated image belongs to the target style.
The traditional way of calculating sytle loss:![](https://github.com/citymonkeymao/neural-style/blob/gan/data/style-gan.png?raw=true)
The new way of calculating style loss:![](https://github.com/citymonkeymao/neural-style/blob/gan/data/style-gan2.png?raw=true)
Results
Imitate Shinkai Makoto Style
Transfered with ~160 high quality style images.![](https://github.com/citymonkeymao/neural-style/blob/gan/data/cmp_manual.png?raw=true)
Imitate Monet(Comparing to CycleGAN)
Imitate Vangogh(Comparing to CycleGAN)
Usage
Download style image set(borrowed from CycleGAN):
bash ./datasets/download_dataset.sh <dataset name>
<dataset name>
could be monet2photo, vangogh2photo, ukiyoe2photo, cezanne2photoDo style transfer
-gan
command specifies using Discriminators to calculate style losses.d_learning_rate
is the learning rate for Discriminators.list_images.sh
helps to list all images in one directory, all files in that directory should not contain space andstyle_image_dir
should not contain~
. You need to play with parameters for different style and size.example
Transfer fj.jpg to vangogh style
bash ./datasets/download_dataset.sh vangogh2photo