search

openai / ebm_code_release

Code for Implicit Generation and Generalization with Energy Based Models
https://sites.google.com/view/igebm
340 stars 64 forks source link

FLAG.objective mode logsumexp and softplus #4

Open ghost opened 4 years ago

ghost commented 4 years ago

Hi, I am curious what is the math background for the logsunexp and softplus case in FLAG.objective.

Thanks.

yilundu commented 4 years ago

Hi, You can find the interpretation of softplus here (appendix): https://arxiv.org/pdf/1811.02486.pdf If you assume that negative samples are drawn from the uniform distribution as opposed from the model distribution than you obtain the logsumexp interpretation.