why has to change to discretisized loss function?

openai / pixel-cnn

Code for the paper "PixelCNN++: A PixelCNN Implementation with Discretized Logistic Mixture Likelihood and Other Modifications"

https://arxiv.org/abs/1701.05517

Other

1.92k stars 436 forks source link

why has to change to discretisized loss function? #38

Open xfwu opened 6 years ago

xfwu commented 6 years ago

Really don't understand why it is needed to change logistic's pdf to cdf (which is sigmoid)... can't it be estimated directly by using logistic's pdf? is it some stablization consideration?

Best

raffy-bekhit commented 5 years ago

cdf includes more information that can be used than the cdf. So to compute the maximum liklihood estimation (MLE) all you have to do in the cdf is calculate the derivative about the wanted point ( let it be y) the higher the derivative ( slope ) the better. the closer y to the mean and the smaller value of the standard deviation -> the higher the derivative is (higher porbability). This all can be calculated by ( cdf(y+e) - cdf(y-e)) where e is a very small value.