Weird Posterior - Problem in Both CosmoMC and Cobaya interface to new Polychord

[vivianmiranda]

Hello!

I have this weird bug that affects newer versions of Polychord - both in my private CosmoMC interface and in the Cobaya new interface. The posterior just disagree drastically with what we get from MCMC (on Polychord 1.09 that was not the case - everything worked fine)

Here is a trivial example I tested on Cobaya

t1.pdf

EXAMPLE_POLYCHORD_2.txt

EXAMPLE_MCMC_2.txt

Thanks for helping me!

[vivianmiranda]

I have seen this bug with multiple dimensions, different datasets and always version > 1.09

[vivianmiranda]

And somehow the Evidence seems correct when comparing to Multinest...just the posteriors are crap..

[vivianmiranda]

Other members of DES collaboration have seen similar problems in the CosmoSiS interface

[williamjameshandley]

Hi @vivianmiranda ,

I ran your cobaya examples on my local setup, and found that they agreed:

Can you confirm that you get this issue on a fresh install of pypolychord from the github repository, having removed all previous versions?

@JesusTorrado Have you seen anything like this before?

Other members of DES collaboration have seen similar problems in the CosmoSiS interface

Could you copy these people in to this discussion?

my private CosmoMC interface

You might like to know that there is now a github repository for CosmoChord, forked from CosmoMC. For your future work, you might like to fork from this to stay up-to-date with any further bug fixes.

[vivianmiranda]

Hello!

It was a fresh install. Troxel has the same mistake in Cosmosis. I will ask him to join.

In CosmoMC I used CosmoChord with some adapts - but I will update.

Different people - in different interfaces - similar problems.

:(

Best Vivian Miranda

[vivianmiranda]

Ah

I used getdist to plot it

Vivian Miranda

[JesusTorrado]

Hi both,

(@vivianmiranda sorry, I've been quite busy. Will come back to the Cobaya issues shortly)

I've seen similar stuff happen one mode in a multimodal pdf has not been discovered: the normalisation of the posterior changes and modes appear larger (contours are larger) than in the actual posterior, but keep the same shape.

So probably a normalization problem (also possibly due to different prior normalisation?)

[vivianmiranda]

Hello!

I dont think is just normalization - because in more complicated setups the posterior look rubbish ( unfortunately I think I deleted my CosmoMC test cases since dowgrading to Polychord 1.09 solved the CosmoMC issues).

Best Vivian Miranda

[williamjameshandley]

I can't attempt to fix this issue without a MWE (at the moment I just have your scripts, which produce the right answer for me on my machine). Please can you tell me:

• What version of PolyChord you are using (number, branch and git hash)
• What version of Cobaya you are using (number, branch and git hash)
• What you are using to plot the posteriors

[vivianmiranda]

Hello

Polychord - ef02bb6d94dca218c7d8daa98e8ac010022e457e - I think this is the most up-dated version

Cobaya - also the latest commit - 1e5506cb662555fbec65c8293d661fffc98572a7.

I use getdist to make the plots.

[matroxel]

I'm not sure what I can add to this concretely right now, since I don't have time to reproduce the testing setup in a way that I could be sure you could replicate, but I was seeing similar issues with reproducing the DES Y1 cosmic shear posterior using polychord relative to multinest within cosmosis. I was getting consistent evidences between the two (though not within the quoted uncertainty of either), but significantly different contours. I'll add more details when I have time. Also @joezuntz that I discussed it with at the time.

[vivianmiranda]

Oh

I am so sorry! I found the mistake after an afternoon trying to hunt down. GetDist when doing both MCMC and Polychord were extracting 0.3 of burn from both. I think that happened every time.

I am so sorry for open that ticket - I usually am very careful in checking

Best Vivian

[JesusTorrado]

It happens :)

Actually, I had already been thinking of adding a setting to GetDist to produce a warning when trying to remove burn-in from nested-sampled chains. I guess I should.

Cheers!

[williamjameshandley]

No worries @vivianmiranda , I'm glad we resolved this, and I know what to point people to in the future if we see similar results.

@JesusTorrado I would go further than a GetDist warning. Burn-in has no meaning for nested sampling, so it should be disabled/throw an exception in the event of people trying to implement it. I've posted an issue.

@vivianmiranda Do you think this was the cause of Troxel's issue?

[vivianmiranda]

Hello!

Thanks for being so kind! I am not sure about Troxel problems - but we can have my silly mistake in mind if it happens again!

Best Vivian Miranda