search

lorenzkuhn / semantic_uncertainty

MIT License
144 stars 20 forks source link

Computing the semantic entropy #9

Open AIchberger opened 1 year ago

AIchberger commented 1 year ago

Hi @lorenzkuhn,

I have a concern regarding the computation of the semantic entropy as part of the function get_predictive_entropy_over_concepts:

llh_shift = torch.tensor(5.0) aggregated_likelihoods = torch.tensor(aggregated_likelihoods) - llh_shift entropy = - torch.sum(aggregated_likelihoods, dim=0) / torch.tensor(aggregated_likelihoods.shape[0])

In this case, aggregated_likelihoods represents the log likelihoods of the semantic sets: log(p(c|x)), shifted by the constant llh_shift. The entropy is then computed as the average negative log likelihood across the semantic sets, which does not follow the standard concept of entropy -sum[p(c|x)*log(p(c|x))]. Could you elaborate on this in more detail?

ranganathkrishnan commented 7 months ago

Hi @lorenzkuhn, I have the same question on semantic entropy computation in the repo as compared to the description in the paper. Can you please give intuition behind the implementation in a different way as proposed in the paper?

Thanks!