Open CarlosNacher opened 3 weeks ago
Hello, I will number the problems you have mentioned so it is easier to answer:
False positive "circles" I will look into this. It seems like a corner case.
Sparse distribution of scores As I understand it, score distribution in the case of EfficientAD can be very sparse because of the inner normalization of anomaly maps done on good images only, so some defects might cause very high scores.
Heat maps look strange
I have tried to run it with normalization_method: none
, and I got this anomaly map which look as expected (trained for 5 epochs):
With normalization_method: minmax
, the maximum score for EfficientAD can be very high, as shown on a plot in question 2, so normalizing with such a high value can cause that strange anomaly map.
pad_maps
and padding
hyperparameters
It is normal, please use default values.
Describe the bug
I know there was a PR that supposedly fixed generl performance problems with EfficientAd model (#2015). However, I am trying (with latest anomalib version) to train EfficientAd on MVTec bottle and I have found several problems.
I relate them sepparately below:
From one hand, the model appears to work well but it gives some strange "circles" as anomalies on the borders of the images.
even in good samples
![even in good samples](https://github.com/openvinotoolkit/anomalib/assets/85699494/72b323b2-30bb-46f2-9e37-38712242d6a1)
From another hand, it sepparates perfectly the distributions (conf matrix perfect) but it seems to have wrong scales, like too sparse (which I think it can be due to #2027), maybe a sigmoid function would fix this?, proofs following:![image](https://github.com/openvinotoolkit/anomalib/assets/85699494/7adf50ee-d0a5-478b-9672-2066f5766fd3)
Also, and I think it is related with the previous point (abaout scales when normalizing), even that the segmentation map looks okay (despite the "border circles"), the anomaly map looks that have always same value constant (#1647)
As said in #1647 I tried both
It looks like the map is being normalized with negative values or something because the more blue pixels are the segmented ones (when the expected behaviour is that the red ones to be the most anomalous). It is strange. As a note: the dataset normalization is retrieved from EfficientAd default transforms (none because in forward pass there is already imagenet norm).
normalization_metod = min_max
( above results) andnormalization_method = none
, giving below results:I have tried all possible combinations of
pad_maps
andpadding
hyperparameters. The best are the default ones (pad_maps = True
.padding = False
).Dataset
MVTec
Model
Other (please specify in the field below)
Steps to reproduce the behavior
Train EfficientAd on MVTec bottle with default configurations.
OS information
OS information:
Expected behavior
EfficientAd to work well in MVTec
Screenshots
No response
Pip/GitHub
pip
What version/branch did you use?
1.2.0dev
Configuration YAML
Logs
Code of Conduct