A Style-Based Generator Architecture for Generative Adversarial Networks

[kingsj0405]

A Style-Based Generator Architecture for Generative Adversarial Networks https://arxiv.org/abs/1812.04948

Related:

1. Image style transfer using convolutional neural networks https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf
2. Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization https://arxiv.org/abs/1703.06868

(가능하다면)

3. HoloGAN: Unsupervised learning of 3D representations from natural images https://arxiv.org/abs/1904.01326

[kingsj0405]

논문세미나1.pdf

Image Style Transfer Using Convolutional Neural Networks, Gatys2016

Task

• Style Transfer
• Original Image + Style Image -> Styled Image

Motivation

• Pre-trained CNN for extracting feature from image
• 중요: 네트워크를 학습시키지 않고 noise를 image로 만들어낸다
• Gram Matrix를 이용해서 중간 역할을 한다!

Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization, Huang2017

Motivation

• gatys는 느리지만 어디든 적용가능
• feed-forward는 빠르지만 single style
• Conditional Instance Normalization: Affine Parameter of Normalization을 style 별로 사용한다

Analysis

• Style Normalization을 하면 batch normalization과 instance normalization의 성능 차이가 사라졌다

Implementation Detail

• AdaIN layer: AdaIN(x,y) = scale(y) ( x - shift(x) / scale(x) )+ shift(y)
• VGG Encoder를 뒤의 레이어들은 transfer learning을 좀 해준다

A Style-Based Generator Architecture for Generative Adversarial Networks, Karras2019

Problem of GAN

• Latent space에 대한 정보가 너무 없다
• 그러니 Generator Architecture를 새로 제안해서 이걸 좀 controllable, interpretable하게 만든다

Style-based Network

• Latent + Original Image + Noise --> Latented Image
• weight(intermediate feature)를 어느 레벨에 넣느냐에 따라서 high level, low level 조정이 가능하다
• Disentanglement가 잘 되지 않을까(high level, low level 사이에서 e.g. 성별과 헤어스타일)
• Stochastic Variation을 모델에서 잘 처리한다
• coarse와 fine layer도 다르다