predictions showing after user annotation

microsoft / aerial_wildlife_detection

Tools for detecting wildlife in aerial images using active learning

MIT License

226 stars 58 forks source link

predictions showing after user annotation #46

Open ctorney opened 3 years ago

ctorney commented 3 years ago

Hi Beni, Another one from me. If I have model predictions for an image, I click clear all and the predictions disappear as expected. If I then hit next and previous I see the model predictions back again. The same behaviour if I label all predictions as something else - if I go next then previous I see the annotations and if I move them I see the old predictions underneath.

Is this expected behaviour? In the old version once an image was annotated if you looked at it again you would only see user annotations and not previous model predictions. Cheers

bkellenb commented 3 years ago

Hi Colin,

This was a kind of expected behaviour since v2.0. The idea was that AIDE would immediately auto-convert predictions into annotations as soon as a new image was displayed. However, nothing would be changed until the user conﬁrmed the predictions. In the latest commit, the behaviour is changed as follows:

Upon click on "Next", the predictions are indeed automatically converted as per slider at the bottom.
A click on "Previous", or else going through a review process, will not do this by default anymore, but just show the annotations made by the user. However, predictions will be auto-converted again as soon as the corresponding slider below the image is adjusted.

I agree that the previous behaviour was a bit confusing and hope that it makes more sense now!

ctorney commented 3 years ago

Hi Beni, I think this makes sense - but just so I'm clear it seems the slider is the key thing here.

Suppose I have a prediction of a wildebeest and it's actually an elephant. I correct the prediction then click next - if I now click previous I'll see my elephant annotation and the old wildebeest prediction. If the slider conversion is set above the prediction certainty if I click next I'll send two overlapping annotations to the db. If I adjust the slider down below the confidence and click next then it'll send only my annotation. Is that right?

bkellenb commented 3 years ago

Hi Colin,

Yes, that should be right. The bottom slider determines the mininum conﬁdence at which a prediction counts as (your) annotation.