Sigmoid VS Exp in AffineCoupling

[MokkeMeguru]

Exp

pros: better reconstruction loss cons: dangerous in training, inference

Sigmoid

pros: safe in training, inference cons: bad reconstruction loss

[MokkeMeguru]

Requirement: To add flags of Affinecoupling to switch Exp and Sigmoid

[MokkeMeguru]

Flag was appended

[MokkeMeguru]

option

• "safe exp" = exp ( value.clip(-15, 15))
• "exp" = exp(value)
• "sigmoid" = sigmoid (value + 2.0)