search

pymc-devs / pymc

Bayesian Modeling and Probabilistic Programming in Python
https://docs.pymc.io/
Other
8.67k stars 2k forks source link

Unconverged chain in dp mixture example #3209

Closed twiecki closed 3 years ago

twiecki commented 6 years ago

image

https://docs.pymc.io/notebooks/dp_mix.html

Probably just rerunning with more tuning steps will fix that.

junpenglao commented 6 years ago

I think there are some problem with the model (mixture model is hard), see attempt here https://github.com/pymc-devs/pymc3/pull/2956

Drwangchao commented 5 years ago

I just met a similar problem. I have post the details at https://discourse.pymc.io/t/questions-about-the-convergence-of-sampling-the-mixture-of-hierarchical-model/2169.

I wonder how to solve this problem. I expect reply. @twiecki @junpenglao

sameshl commented 5 years ago

I am new here and would like to start contributing by taking up this issue.Can somebody link me up with relevant info so that I can get started.

twiecki commented 5 years ago

@sameshl The relevant info should be found in the links above. Try running the NB and doubling the tuning steps and setting target_accept=0.9.

Harivallabha commented 5 years ago

Hi, could I get started on this issue? I'm new to pyMC3 and would like to work on the GSoC, 2019 project on Dirichlet processes :)

junpenglao commented 5 years ago

@Harivallabha sure!

Harivallabha commented 5 years ago

Hi, sorry for the delay. Had mid-semester examinations for the past two weeks :)

I tried increasing the # of tuning steps, and setting target_accept to 0.9. It still diverges. I've included a couple of snapshots of the results. Here's a link to my notebook where I tried out a few other configurations as well: https://github.com/Harivallabha/DP_pyMC3-Docs_Issue/blob/master/Unconverged_DP_Chain.ipynb

I'll now try reparameterizing it, and use advi+adapt_diag for initializing the sampler. Hopefully a proper reparameterization should do the trick.

Screenshot (605)

Screenshot from 2019-03-18 15-46-36

canyon289 commented 5 years ago

@Harivallabha I mean this in the kindest way, posting screenshots of a notebook is not a great way to share code. I would suggest a minimal example, or even better a github gist or repo where was can inspect the results and download the code

Harivallabha commented 5 years ago

@canyon289 That makes sense, thank you. Edited it :)

Harivallabha commented 5 years ago

Hi, I tried a few other things but seem to be unable to fix the divergences. Here's a link to my notebook outlining my progress: https://github.com/Harivallabha/harivallabha.github.io I've also included in it, a compilation of resources and notes that I found useful while trying to understand and fix this issue (for the benefit of any newcomer looking to gain some insight about the topic). Any suggestions would be deeply appreciated :) Thanks!

twiecki commented 5 years ago

Thanks for your work on this! Where exactly is the funnel occurring, do you think it can be reparameterized? I'm not sure ADVI is more than just masking the problem.

ricardoV94 commented 3 years ago

Closing this due to https://github.com/pymc-devs/pymc3/pull/3724. There are still problems with divergences, but the sampler is converging at least.

If anyone wants to reopen this issue, it should be done in https://github.com/pymc-devs/pymc-examples