search

yaojieliu / ECCV20-STDN

Source code for ECCV 2020 paper: On Disentangling Spoof Trace for Generic Face Anti-Spoofing
http://cvlab.cse.msu.edu/project-face-anti.html
Other
150 stars 35 forks source link

Calculation of pixel wise loss. #20

Open MatZlo opened 3 years ago

MatZlo commented 3 years ago

Issue:

https://github.com/yaojieliu/ECCV20-STDN/blob/c79f1f8c615d2b8471b3df29da881bb18dd54c90/train.py#L108

As far as I can tell the current pixel loss compares the ground truth spoof traces with spoof traces generated from real live images. Should it not be the following instead? (Changed selection of traces_a) Please correct me if I'm wrong, and thank you for your time.

Original: traces_a[:bsize,...] - Selects the spoof traces from live images

pixel_loss = l1_loss(traces_a[:bsize,...], tf.stop_gradient(trace_warp))

Proposed change: traces_a[bsize:,...] - Would select the spoof traces from the synthetic spoof images

pixel_loss = l1_loss(traces_a[bsize:,...], tf.stop_gradient(trace_warp))

In your paper you mention that you calculate per pixel loss of spoof traces like this: image_pixel_loss

The flow of data according to train.py:

  1. The original creation of the synthetic live and synthetic spoof:
  recon1 = (1-s)*img - b - tf.image.resize_images(C, [imsize, imsize]) - T
  trace = img - recon1
  trace_warp = warping(trace[bsize:,...], reg, imsize)
  synth1 = img[:bsize,...]+ trace_warp

img - dimenstions [2 * bsize, ...] Where first half are real live images and other half is real spoof images. synth1 - dimenstions [bsize, ...] Where all are synthetic spoof images.

  1. You concat live images with synthetic spoof images for batch normalisation. Assuming that tf.cond() always picks img_a2 for simplicity.
  img_a2 = tf.stop_gradient(tf.concat([img[:bsize,...], synth1],axis=0))
  dec = tf.greater(tf.random.uniform([1],0,1)[0],0.5)
  img_a = tf.cond(dec,lambda: img_a1, lambda: img_a2)
  M_a, s_a, b_a, C_a, T_a = Gen(img_a, training_nn=training_nn, scope='STDN')
  traces_a = s_a*img + b_a + tf.image.resize_images(C_a, [imsize, imsize]) + T_a

_imga2 - dimensions [2 bsize, ...] Where all first half is real live images and the other half are synthetic spoof images. _tracesa - dimensions [2 bsize, ...] Where all first half are spoof traces from live images, the other half are spoof traces from the synthetic spoof images.

  1. The loss step.

    pixel_loss = l1_loss(traces_a[:bsize,...], tf.stop_gradient(trace_warp))

_tracesa[:bsize,...] - Selects the spoof traces from live images. _tracesa[bsize:,...] - Would select the spoof traces from the synthetic spoof images.

sherlockers commented 3 years ago

Yes, I also found this problem