search

cvlab-epfl / detecting-the-unexpected

Detecting the Unexpected via Image Resynthesis
Other
56 stars 10 forks source link

Why not use object detection to achieve this goal? #16

Open kaigelee opened 2 years ago

kaigelee commented 2 years ago

I have a question, why not use object detection to achieve this goal?

GabrielDornelles commented 2 years ago

I believe the idea here is to detect things out of the supervised environment.

In that case, object detection (as a supervised method) doesnt fit in such a task. Just imagine different scenarios where things that were not under your radar at training appears, that could be in the streets, at PCBs, in a sea image, or many others cases. You simple want to see if there's something unexpected or not in your images

kaigelee commented 2 years ago

I believe the idea here is to detect things out of the supervised environment.

In that case, object detection (as a supervised method) doesnt fit in such a task. Just imagine different scenarios where things that were not under your radar at training appears, that could be in the streets, at PCBs, in a sea image, or many others cases. You simple want to see if there's something unexpected or not in your images

Thank you very much for your answer, I may not have described my problem clearly. What I want to ask is why we don't use object detection (not supervised methods, but Open world object detection methods) to detect something unexpected, but semantic segmentation?

GabrielDornelles commented 2 years ago

I'm not really sure, and I don't speak by the authors.

In my eyes, It could yes use a open world object detection, but that requires me to do the whole process of building and cleaning a dataset.

Also, if the situation is not open world with these already existent datasets, it would require me to label and supervise the whole thing.

Iimagine you're analysing some different types of objects in the sea for example. You want to see some fishes or animals, and you would like to detect the anomalies that could appears there. You're not aware of every type of thing/object that could appear there, but you want the sea to be clean without these objects moving there.

The thing is, with object detection, you're limited to use a dataset and supervise it to draw a box around the objects, you would either detect and unknown object as one of your classes, or not detect at all (most likely I think). Now segmentation does give you a probability map, and thus you can work with it pixel by pixel and build maps.

On their work they use that map that does ignore the unknown object to re-create the image, so the anomalies come in evidence when comparing the original with the re-created image.

Nothing really stops you to try to work with a object detection pipeline to perform such task, but its not really clear (at least for me) how would you do that.

adynathos commented 2 years ago

Object detection methods may be a valid approach for this task, but they were not evaluated in this particular paper which is focusing on comparing images and finding anomalous pixels.

One could speculate that if we don't know what the anomalies are, how would we know how to divide them into instances - ie 2 anomalies next to each other. However that is only speculation and not tested in experiment.