FGS "Use logit mode if computing loss w.r.t. output scores rather than softmax"

facebookarchive / adversarial_image_defenses

Countering Adversarial Image using Input Transformations.

Other

489 stars 75 forks source link

FGS "Use logit mode if computing loss w.r.t. output scores rather than softmax" #4

Closed ysharma1126 closed 6 years ago

ysharma1126 commented 6 years ago

So mode needs to be set to FGSMode.LOGIT, if I want to use the logits? If mode is 'None', the cross-entropy loss is used, if mode is 'FGSMode.CARLINI', the output is updated, then the FGSMode.LOGIT loss is used.

Not entirely sure what's going on. If I wanted to use the logits, and use the xent loss, what mode should I set?

lvdmaaten commented 6 years ago

You cannot use a cross-entropy loss on logits: a cross-entropy loss operates on probabilities (obtained via, for instance, a softmax operation), not on unnormalized scores. The different mode behaviors are defined here.