search

kzhai / PyNPB

Non-parametric Bayesian in Python, including Indian buffet process (IBP), hierarchical Dirichlet process (HDP).
73 stars 32 forks source link

Likelihood decrease when fitting HDP with gibbs sampling (ugs.py) #3

Open dtrckd opened 8 years ago

dtrckd commented 8 years ago

Hi,

I'am experiencing your NP prior, in order to try it in some relational model.

When running ugs HDP over a NIPS 2012 text corpus, the log-likelihood decrease with iterations. Did you observe some convergence or a likelihood improvement in your experiments ?

Output that I get:

sampling in progress 10% total number of topics 3, log-likelihood is -608665.500427 sampling in progress 20% total number of topics 4, log-likelihood is -611818.654787 sampling in progress 30% total number of topics 4, log-likelihood is -618855.827824 sampling in progress 40% total number of topics 4, log-likelihood is -621922.608209 sampling in progress 50% total number of topics 5, log-likelihood is -627783.435882 sampling in progress 60% total number of topics 5, log-likelihood is -631405.448569

kzhai commented 8 years ago

There were some likelihood terms with hyper-parameters missing, I think I just corrected it, please check if that works now. Thanks for reporting the issue.

Best, Ke

dtrckd commented 8 years ago

Hello, thanks for your reactivity.

I updated the changes, it seems to decrease again:

sampling in progress 3% total number of topics 2, log-likelihood is -598247.291138 sampling in progress 6% total number of topics 3, log-likelihood is -601833.297795 sampling in progress 9% total number of topics 2, log-likelihood is -603406.460173 sampling in progress 12% total number of topics 4, log-likelihood is -605434.257627 sampling in progress 15% total number of topics 3, log-likelihood is -607361.818930 sampling in progress 18% total number of topics 4, log-likelihood is -612521.745021

kzhai commented 8 years ago

Huh... Let me check and get back to you later.

Best, Ke