search

gudovskiy / cflow-ad

Official PyTorch code for WACV 2022 paper "CFLOW-AD: Real-Time Unsupervised Anomaly Detection with Localization via Conditional Normalizing Flows"
https://arxiv.org/abs/2107.12571
BSD 3-Clause "New" or "Revised" License
234 stars 58 forks source link

Negative Log Sigmoid Log Likelihood Loss? #23

Open marco-rudolph opened 2 years ago

marco-rudolph commented 2 years ago

Hi Denis! According to your code, you use some "negative log sigmoid log likelihood loss": https://github.com/gudovskiy/cflow-ad/blob/4d6ec4719e836422c5253a68c1150ac172d8000d/train.py#L76 https://github.com/gudovskiy/cflow-ad/blob/4d6ec4719e836422c5253a68c1150ac172d8000d/train.py#L78 with https://github.com/gudovskiy/cflow-ad/blob/4d6ec4719e836422c5253a68c1150ac172d8000d/train.py#L18

What is the motivation behind using this kind of loss?

Thanks in advance, Marco

gudovskiy commented 2 years ago

@marco-rudolph well, flows can minimize KL-divergence up to the constant or, alternatively, can have log-likelihoods larger than 0.0 which is not desirable. So, to prevent this and to avoid negative loss I added sigmoid. In practice, it can help to improve performance a bit.

marco-rudolph commented 2 years ago

Why are likelihoods larger than 0.0 not desireable? Negative loss itself should not be a problem as long as it does not lead to instabilities which is not really the case here. But it is an interesting finding that this improves the performance :)