about entorpy

[albb762]

hi. Thank you for share your code. But I try it with some toy data. I found out, If we want to assign a high score to the data which has uniform P(y) and skew P(yx), we should use entropy(py, pyx), instead of entropy(pyx, py). And here is the code from openai: kl = part * (np.log(part) - np.log(np.expand_dims(np.mean(part, 0), 0))) https://github.com/openai/improved-gan/blob/master/inception_score/model.py

[kmaeii]

hi. Thank you for share your code. But I try it with some toy data. I found out, If we want to assign a high score to the data which has uniform P(y) and skew P(yx), we should use entropy(py, pyx), instead of entropy(pyx, py). And here is the code from openai: kl = part * (np.log(part) - np.log(np.expand_dims(np.mean(part, 0), 0))) https://github.com/openai/improved-gan/blob/master/inception_score/model.py

Hi, I have the same confuse with you, so, should i replace the 'entropy(py, pyx)' to 'kl = part * (np.log(part) - np.log(np.expand_dims(np.mean(part, 0), 0)))'. In my opinion, this is not equal.

[sbarratt]

In the original paper ( https://papers.nips.cc/paper/6125-improved-techniques-for-training-gans.pdf on page 4), it is ‘entropy(pyx, py)’, or the KL divergence between p(y|x) and p(y).

Shane

On Sun, Dec 22, 2019 at 12:33 AM kmaeii notifications@github.com wrote:

hi. Thank you for share your code. But I try it with some toy data. I found out, If we want to assign a high score to the data which has uniform P(y) and skew P(yx), we should use entropy(py, pyx), instead of entropy(pyx, py). And here is the code from openai: kl = part * (np.log(part) - np.log(np.expand_dims(np.mean(part, 0), 0))) https://github.com/openai/improved-gan/blob/master/inception_score/model.py

Hi, I have the same confuse with you, so, should i replace the 'entropy(py, pyx)' to 'kl = part * (np.log(part) - np.log(np.expand_dims(np.mean(part, 0), 0)))'. In my opinion, this is not equal.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/sbarratt/inception-score-pytorch/issues/9?email_source=notifications&email_token=AB7LUGM2PPKMXKWMQAQSK5TQZYLLHA5CNFSM4FN5IFL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHO4GGI#issuecomment-568181529, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB7LUGKT6YZNYCVY3INHRN3QZYLLHANCNFSM4FN5IFLQ .

[bomtorazek]

scipy.stats.entropy uses the KL divergence if two distributions are given.

"If qk is not None, then compute the Kullback-Leibler divergence" You can check the document below, https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.entropy.html