search

WenzhengZhang / EntQA

Pytorch implementation of EntQA paper
MIT License
60 stars 12 forks source link

Probability of predicted entities decreases with increase in number of candidates #9

Closed HongShihan closed 1 year ago

HongShihan commented 2 years ago

Hello, thank you for your very interesting work. I noticed that the final probability of the detected entities tends to decrease when i increase the number of candidates, which sometimes causes the result to be less accurate with increased k as the probability falls below the threshold. Is this normal behavior? Possibly due to the re-ranking function?

WenzhengZhang commented 1 year ago

Hi, Sorry for the late reply! Your assumption make sense because we compute the entity probability by taking softmax over all candidates. In this paper, we tune the threshold for my specific number of candidates. So we may need to use the dynamic threshold or tune a more suitable threshold for more candidates. Also, we note that you can also choose not to use the re-rank probability to compute the mention probability, which we found doesn't hurt the performance and prevents the issue you mentioned here.