search

abhijitbendale / OSDN

Code and data for the research paper "Towards Open Set Deep Networks" A Bendale, T Boult, CVPR 2016
Other
245 stars 57 forks source link

fit_high() function gives high scores for higher distances while low scores for low distances #7

Closed sadimanna closed 6 years ago

sadimanna commented 6 years ago

In the paper "Towards Open Set Deep Networks" it is mentioned, that we have to do per class Weibull fit using FitHigh function. However, in the documentation html files it is written that FItHigh should be used if the data is such that larger is better, which I suppose is referred to large distances from the mean activation vector.

Using FitHigh function, gives larger scores for larger distances, but according to the paper, should it be the opposite? What I meant to say is that, should, we get low scores for larger distances from the mean activation vector?

And if I am not wrong, shouldn't FitLow function be used in place of FitHigh?

This is what I obtained

import libmr as mr meta = mr.MR() meta.fit_high(sorted_dist[-100:],100) meta.w_score(0.0) 0.0 meta.w_score(10.0) 0.21476801948365665 meta.w_score(20.0) 0.9963975817523232 meta.w_score(30.0) 0.9999996470632248 meta.w_score(40.0) 0.9999999999978881 meta.fit_low(sorted_dist[-100:],100) meta.w_score(0.0) 1.0 meta.w_score(10.0) 0.8115743914686449 meta.w_score(20.0) 1.665564614727888e-05 meta.w_score(30.0) 0.0 meta.w_score(40.0) 0.0

I am also attaching the plot of the sorted distances of all the 1011 correctly classified training examples of only one class, from the mean activation vector of that class. figure_1

o0o6666 commented 5 years ago

@sadimanna I have same question. Did you figure out the answer?

XiaoyanQian commented 3 years ago

Hi Guys, I also have the same problem. Did you have any answers?

sadimanna commented 3 years ago

@o0o6666 @XiaoyanQian I did not get any reply from the author, but I guess using the FitLow function suffices the need. I wouldn't comment much as I am not currently working on the topic and won't like to misguide any of you. But I would suggest you explore the FitLow function a bit and validate if the end results conform to the ground truth.

sfeilaryan commented 3 months ago

I believe you choose to fit high so that you can choose to have your fit accommodate a the fit_size highest values - distances in this case - so that you get more information on the vectors that are true to the class but far from the MAV, as it is perhaps more valuable than the information of the vectors close to the MAV.