Multiplication of prior with log likelihood - Code not found !!

[jeethesh-pai]

@BlackHC Can you please tell me where you are multiplying the prior with the log_prob of the logits during evaluation.

As mentioned in the line 12: We have to multiply the probability of the feature vector with the prior obtained from training. I saw a part where you perform calulcation of log probability

https://github.com/omegafragger/DDU/blob/f597744c65df4ff51615ace5e86e82ffefe1cd0f/utils/gmm_utils.py#L79

https://github.com/omegafragger/DDU/blob/f597744c65df4ff51615ace5e86e82ffefe1cd0f/metrics/uncertainty_confidence.py#L16 and https://github.com/omegafragger/DDU/blob/f597744c65df4ff51615ace5e86e82ffefe1cd0f/evaluate.py#L211

but didnt actually find the code snip where you were multiplying with the prior. Is it required or is it enough to just calculate the logsumexp() of the logits obtained.

Thanks @BlackHC

[BlackHC]

Thanks!

I think the assumption is that the dataset is balanced and the prior is approx equal and hence can be dropped as a constant shared factor.

Jishnu might know more?

On Thu, Nov 17, 2022, 11:29 Jeethesh Pai Umesh @.***> wrote:

@BlackHC https://github.com/BlackHC Can you please tell me where you are multiplying the prior with the log_prob of the logits during evaluation. [image: grafik] https://user-images.githubusercontent.com/75034628/202408223-1acf274d-23a7-494f-8965-153f1a2e346e.png

As mentioned in the line 12: We have to multiply the probability of the feature vector with the prior obtained from training. I saw a part where you perform

https://github.com/omegafragger/DDU/blob/f597744c65df4ff51615ace5e86e82ffefe1cd0f/metrics/uncertainty_confidence.py#L16 and

https://github.com/omegafragger/DDU/blob/f597744c65df4ff51615ace5e86e82ffefe1cd0f/evaluate.py#L211

but didnt actually find the code snip where you were multiplying with the prior. Is it required or is it enough to just calculate the logsumexp() of the logits obtained.

Thanks @BlackHC https://github.com/BlackHC

— Reply to this email directly, view it on GitHub https://github.com/omegafragger/DDU/issues/6, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFSBYFYXZ3BD3YBBDSRLRDWIYJKTANCNFSM6AAAAAASDISNYY . You are receiving this because you were mentioned.Message ID: @.***>

[jeethesh-pai]

Yea, thats true. It makes sense to avoid that if the dataset is balanced. But do you know how to change the code if it was'nt

[omegafragger]

Hi, sorry for the delayed reply on this.

The way I'd do it is compute log q(y) where q(y) is the probability of getting class y. From the dataset it is simply (# samples in class y / total # samples).

The log q(z|y) is the log prob from the Gaussian containing mu_y and sigma_y as parameters.

Both the above tensors are K dimensional where K is the number of classes. You could sum these tensors up and then take a logsumexp of the resulting tensor over classes. That should give you log q(z).

Hope this helps! Apologies again for the delay in replying.

[jeethesh-pai]

Hi,

Thanks Jishnu, That was exactly what I did in my case. Thanks for your confirmation. Do you know if we can take any valuable decision like the amount of uncertainity (in percentage) or smthing with the q(z) value we find. I would like to see what is the uncertainity value of the networks current prediction (say it predicts class A as its output but actual Ground Truth was supposed to be B). Can I deduce that the network was already x% uncertain in predicting A whereas it had only y% uncertainty in predicting class B?