Planck likelihood error causing Montepython+Polychord to abort

[ivandebono]

Hi Will,

I'm not entirely sure that this is a Polychord issue, so bear with me.

I'm running Montepython with Polychord (--method PC) with Planck 2018 data and likelihoods (plc-3.01). My input parameter file works just fine with Montepython Metropolis-Hastings sampling. But running with Polychord produces an error which then causes Montepython to abort the run before all the live points have been produced.

Here is the error message:

File "/montepython_public/montepython/MontePython.py", line 40, in <module>
    sys.exit(run())
  File "/montepython_public/montepython/run.py", line 44, in run
    sampler.run(cosmo, data, command_line)
  File "/montepython_public/montepython/sampler.py", line 51, in run
    pc.run(cosmo, data, command_line)
  File "/montepython_public/montepython/PolyChord.py", line 377, in run
    polychord_run(loglike, nDims, nDerived, settings, prior)
  File "/.local/lib/python2.7/site-packages/pypolychord-1.16-py2.7-linux-x86_64.egg/pypolychord/__init__.py", line 231, in run_polychord
    settings.seed)
  File "/.local/lib/python2.7/site-packages/pypolychord-1.16-py2.7-linux-x86_64.egg/pypolychord/__init__.py", line 192, in wrap_loglikelihood
    logL, phi[:] = loglikelihood(theta)
  File "/montepython_public/montepython/PolyChord.py", line 362, in loglike
    logl = sampler.compute_lkl(cosmo, data)
  File "/montepython_public/montepython/sampler.py", line 770, in compute_lkl
    value = likelihood.loglkl(cosmo, data)
  File "/montepython_public/montepython/likelihood_class.py", line 1048, in loglkl
    lkl = self.clik(tot)[0]
  File "lkl.pyx", line 89, in clik.lkl.clik.__call__
clik.lkl.CError: clik_compute(../src/clik.c:391)::ForwardError
  distribution_lkl(../src/minipmc/distribution.c:219)::ForwardError
    lklbs_lkl(../src/lklbs.c:219)::ForwardError
      simall_lkl(../src/simall/clik_simall.c:48)::Error -1233 (multipole EE 3 too large (got 0.42259 expected <0.3))

In an attempt to resolve the issue, I tried using an extremely low number of live points. Depending on the data set (TTTEEE or EE), I tried nlive = 20 or 50. This allowed all the live points to be output, and for the sampling to start.

I'm not quite sure what is going on here. The error message seems to point to some unphysical region of the prior space, but why is that a problem? Shouldn't Polychord (or Montepython) just discard that point and try another one?

Why should the number of live points change anything?

If anyone else has experienced this issue, I'd be grateful for any hints.

[williamjameshandley]

Hi @ivandebono. This is a planck likelihood/montepython issue. However it will in general only crop up when using nested sampling due to the fact that it explores the deep tails of a distribution. I believe that @lukashergt has encountered this, and that when using CAMB/CosmoMC that this is a silent error, so it might be worth contacting the montepython crew to see how they advise excluding these points from the sampling distribution.

Sorry I can't be of more help. I'd advise raising an issue on the montepython repository, and looping me into that conversation.

[ivandebono]

Thank you.

You're right: the error occurs on CAMB/CosmoMC, but CosmoMC just keeps on running.

[williamjameshandley]

A very similar issue was also addressed in cobaya by @JesusTorrado, who may be able to provide further advice.

[ivandebono]

Thank you.

I used a try/except block for the CMB likelihood in likelihood_class.py, assigning a value of 0 to lkl when the exception occurs. It's a crude solution but it's the only one that occurred to me.

[williamjameshandley]

The value you assign in these cases should be 'log zero' (typically -numpy.inf, -1e30, or -1e300). You will likely see some very odd behaviour if you assign the log likelihood a value of 0.

[JesusTorrado]

Hi @williamjameshandley and @ivandebono

You should open an issue in MontePython to catch that error.

Also, as @williamjameshandley says, go for some approx of log(0)