deeplearning - 4/w4

[getElementsByName]

Special applications: Face recognition & Neural style transfer

• Discover how CNNs can be applied to multiple fields, including art generation and face recognition. Implement your own algorithm to generate art and recognize faces!
• Face Recognition
  • https://www.youtube.com/watch?v=wr4rx0Spihs
• Neural style transfer
  • https://www.youtube.com/watch?v=aQcj13gX73Y

Face Recognition

What is face recognition?

• Face Verification와 Face Recognition 비교

Face Verification

• input 이미지가 특정 사용자인지 아닌지 판단
• 예) 주어진 이미지가 유재석인가?

Face Recognition

• input 이미지가 누구인가?
  • 누구를 판단하기 위한 데이터베이스가 전제
• 예) 주어진 이미지가 누구인가?

One Shot Learning

• For most face recognition applications you need to be able to recognize a person given just one single image.
  • wiki: https://en.wikipedia.org/wiki/One-shot_learning
• solution: similarity
  • This allows us to learn to recognize a new person given just a single image of that person(one-shot learning problem).

Siamese Network

• deep layer에 feature map, embedding 비교

• Siamese twin: https://en.wikipedia.org/wiki/Chang_and_Eng_Bunker
• Siamese connection: https://en.wikipedia.org/wiki/Siamese_connection

Triplet Loss

• Anchor, Positive, Negative 를 하나의 세트로 구성
  • Positive: Anchor와 동일 인물
  • Negative: Anchor와 다른 인물
• max(..., 0): margin 만큼 차이가 발생하면 이후의 차이는 고려하지 않음
• (the idea of a blank net or deep blank is a very popular way of naming algorithms in the deep learning world)

Face Verification and Binary Classification

• So, this is a way to treat face recognition just as a binary classification problem. And this is an alternative to the triplet loss for training a system like this.
• 유사도를 측정하는 여러가지 variation이 존재
  • 예) Chi-square similarity

Neural Style Transfer

What is neural style transfer?

What are deep ConvNets learning?

• These are the nine patches that cause one hidden unit to be highly activated. And then each grouping, this is a different set of nine image patches that cause one hidden unit to be activated.
  • train hidden units in layer 1, they're often looking for relatively simple features such as edge or a particular shade of color.
  • later units are actually seen larger image patches

    • Visualizing what ConvNets learn: https://cs231n.github.io/understanding-cnn/

Cost Function

• 새로 생성하는 input 이미지가 학습하는 파라매터

Content Cost Function

• 중간 hidden layer의 activation 값 차이

Style Cost Function

• 동일 layer의 필터간 상관관계

• Gram Matrix (Style Matrix)

• correlation 으로 표현하는 부분은 수학적으로 엄밀히 unnormalized cross covariance 임. 직관적인 표현을 위해 correlation으로 표현

  • 평균을 빼는 항이 없기 때문

• 각 layer의 Style Matrix 간 차이의 Frobenius norm을 구하고 모두 합한다.

1D and 3D Generalizations

• 2D 데이터 뿐만 아니라 1D/3D 데이터도 CNN을 적용할 수 있음
• 데이터 차원에 대한 Generalizations

Practice questions

Quiz: Special applications: Face recognition & Neural style transfer

Programming assignments

Programming Assignment: Art generation with Neural Style Transfer

content cost

style cost

Generated image correlated with content image

Now, we initialize the "generated" image as a noisy image created from the content_image.

• The generated image is slightly correlated with the content image.
• By initializing the pixels of the generated image to be mostly noise but slightly correlated with the content image, this will help the content of the "generated" image more rapidly match the content of the "content" image.

Programming Assignment: Face Recognition

기타

Kaggle 실습

• https://www.kaggle.com/wonhongyoo/artistic-neural-style-transfer-using-pytorch?scriptVersionId=41463482

tensorflow turotials / generative / neural style transfer

• https://www.tensorflow.org/tutorials/generative/style_transfer