search

FDUCSLG / PRML-2019Spring-FDU

Course Website for PRML Spring 2019 at Fudan University
https://zfhu.ac.cn/PRML-Spring19-Fudan/
19 stars 31 forks source link

Can we use the true distribution to evaluate how well our estimation is? #5

Open ichn-hu opened 5 years ago

ichn-hu commented 5 years ago

No, you should not do it!

Because you should pretend that you don't know the true distribution even if you can get to know it during your estimation. You should use the true distribution as a black box, and the only way you can touch it is to observe its output by get_data function.

In machine learning, peaking into the true distribution or the testing dataset during you estimation or training time is very wrong, and it is kind of like cheating. So please don't do that.

This is another question asked in person to me, I have to repost it here to make everybody aware of it. So in order to make our life easier, from now on please directly ask your question using this issue system!

xlplayer commented 5 years ago

So can we utilize the true distribution after the estimation process to test how well our models are? I think we can evaluate how well our estimation is after we finish the density estimation. What we can not do is using the true distribution to optimize our models.

ichn-hu commented 5 years ago

Yes, you ar more than welcomed to use the true distribution after all of your estimation.