search

marian-nmt / marian

Fast Neural Machine Translation in C++
https://marian-nmt.github.io
Other
1.25k stars 233 forks source link

Why is the cross entropy calculated by marian-scorer negative？ #295

Open zhangxinlu16 opened 5 years ago

zhangxinlu16 commented 5 years ago

When I use marian-scorer to calculate cross-entropy for each sentence pair, why i get negative score for each sentence pair cross entropy?

kpu commented 5 years ago

log

zhangxinlu16 commented 5 years ago

log

According to the definition of cross entropy, the result should be non-negative. Is marian-scorer's result a negative cross entropy?

emjotde commented 5 years ago

Hi, The per-sentence score is just a sum of log probabilities, so technically a negative cross-entropy. Taking exp(score) you would get the total sentence probability in normal probability space between 0 and 1.

zhangxinlu16 commented 5 years ago

Hi, The per-sentence score is just a sum of log probabilities, so technically a negative cross-entropy. Taking exp(score) you would get the total sentence probability in normal probability space between 0 and 1.

Thanks for your reply!