Plot of contour is getting cut.

[karimpsi22]

Hello,

I am facing an issue regarding plotting my chains. For some reason my posterior contour is cut at some values. I've been playing with the number of bins and smooth value in order to fix it, likewise, I tried the kde smoothing by changing the values of kde but still obtaining the same cut.

Besides, using other code like getdist the plot is not presenting such cut.

Here I leave the settings for my plot:

np.random.seed(0)
c = ChainConsumer()

c.add_chain(np.array([params_cose_lcdm.Omega_m, params_cose_lcdm.S_8]).T, 
            parameters=["$\Omega_{\\rm m}$", "$S_8$"], weights=(s_cosebis_lcdm.weights), name="$\Lambda$CDM")

c.add_chain(np.array([params_cmb.omegam, params_cmb.S8]).T, 
            parameters=["$\Omega_{\\rm m}$", "$S_8$"],weights=p_cmb.weights, name="Planck")

c.configure(kde=[2, 2], sigmas=[1, 2], diagonal_tick_labels=False,
             spacing=0.0, colors=["#292421", "#EE82EE"]
           , shade=[True,True], shade_alpha=[0.5,0.5], bar_shade=[True, True]
           ,linestyles=["-", "-"], legend_kwargs={"loc": "lower right", "fontsize": 10},
            )
#c.configure(bins=[20,20], smooth=[20,20])

fig = c.plotter.plot(extents=[(0.25, 0.35),(0.7, 0.9)])

fig.set_size_inches(1 + fig.get_size_inches())

I also attach the output plot (the grey one is cut). I really want to use ChainConsumer to report this plot.

My chains were obtained by using Montepython + Multinest. They are loaded by using getdist.

Could anyone help me out on what is this happening? :-)

Thanks in advance, Karim

[renecotyfanboy]

Hi !

Sorry for this late answer... I also encountered this behaviour once when using CC v0.34 (see orange chain below), but I wasn't aware that it was still an issue in v1.x.x. From what I understood, it comes from the KDE smoothing. Could you tell me if it still occurs when you disable it? Are you willing to share the data as a csv or similar, so I can cross-check with my own?

@Samreay any thought before I dig in the KDE's inner working?

[Samreay]

The normal thing I'd suggest is to look at the extents, but I see this is already configured. Thanks for the tag too, didn't see this get raised at all. @karimpsi22, any chance you can share those numpy arrays so we can try and reproduce?

[renecotyfanboy]

On my side, the weird cut might be due to badly sampled data. When plotted with corner.py, there's a second mod which might trigger weird behaviour when kde-smoothed. I linked the data below.

Edit : This behavior is occuring in my old 0.3.4 code and I was unable to remove it with KDE 2nd Edit : I just ported this to CC v1.0.2 and this behavior seems to come from my data which is clipped

bug_cc.zip

[karimpsi22]

Hi both,

if I disable the kde smoothing, and then I set:

c.configure(bins=10, smooth=5, diagonal_tick_labels=False, ...)

I am still getting a cut in the contour plot (see image).

cc_cut.pdf

Sure, I am sharing the data from this last plot. data_for_cc.zip

Cheers, Karim

[Samreay]

Hey @karimpsi22, any chance I can get you to export that data as a numpy array or pandas dataframe? Im not sure how to load it in

[karimpsi22]

Sorry for not showing how I manage the data. As I mentioned before, I am using getdist to export and process the data. Here is how I loaded the data and generated the latest plot:

import os
import numpy as np
from getdist.mcsamples import loadMCSamples
from chainconsumer import ChainConsumer

path_file = os.path.abspath('./COSEBIS_EMU_NS_DS_with_baryons_more_accurate/NS/COSEBIS_EMU_NS_DS_with_baryons_more_accurate')
sample_load = loadMCSamples(path_file)
params = sample_load.getParams()

samples = sample_load.copy(settings={'mult_bias_correction_order':1,
                       'smooth_scale_2D':-1.0, 
                       'smooth_scale_1D':-1.0}
                          )
np.random.seed(0)
c = ChainConsumer()
c.add_chain(np.array([params.Omega_m, params.S_8]).T, 
            parameters=["$\Omega_{\\rm m}$", "$S_8$"], weights=(samples.weights))

c.configure(bins=[10], smooth=[5],  diagonal_tick_labels=False,
    #kde=[2], sigmas=[1,2], diagonal_tick_labels=False,
             spacing=0.0, colors=["#292421"]
           , shade=[True], shade_alpha=[0.5], bar_shade=[True]
           ,linestyles=["-"],
            )

fig = c.plotter.plot(extents=[(0.0, 0.6),(0.6, 1.0)])

fig.set_size_inches(1 + fig.get_size_inches())
plt.savefig('cc_cut.pdf', bbox_inches='tight', transparent=True, facecolor='white')

Just by using the function params."here the cosmo-param" then you would obtain the numpy array of the cosmo parameter.

Cheers, Karim

[Samreay]

tmp.ipynb.gz

Notebook attached. Example reproduced using the latest version of chainconsumer. This clipping is caused by having too few bins in your overrides. Here's what the data looks like with no smoothing at all:

It does have a very well defined plateau at ~om=0.12, for whatever reason that might be (do you have any priors boundaring om around this mark?)

With the default parametrisation, you can see there's no clipping, but the contour still has a vertical falloff.

And its when you apply the 10 bins that things get funky and clipped.

I'd recommend:

• Getting about 10x the number of samples so you need to smooth less
• Investigating the model to see if there's any reason behind the posterior surfaces Omega_m cliff

[karimpsi22]

I just made the same plot but now from getdist and the cut disappear (see image).

The code for generating the plot from getdist:

import getdist.plots as gdplt
gdplot = gdplt.get_subplot_plotter(width_inch=5)
gdplot.triangle_plot([samples], ['Omega_m','S_8'], 
                legend_labels = [], 
                line_args = [{'lw':1., 'color':'deepskyblue', 'ls':'-.'}], 
                contour_colors = ['deepskyblue'], 
                filled=[True], contour_ls= ['-.'], contour_lws=[1.])

plt.savefig('plot_with_getdist.pdf', bbox_inches='tight')  

plot_with_getdist.pdf

I wonder if loading the data from getdist is causing somehow a bug with ChainConsumer.

Cheers, Karim

[Samreay]

This is because getdist uses a different KDE methodology by default, which I don't have in ChainConsumer. It's also not something I want to add in chainconsumer, because the KDE in getdist has some assumptions about boundary effects (good ones in 99% of cases) that set its behaviour, while my philosophy is more a "If youre chains dont have enough samples, I want you to know about it". This is CC letting you know you should get more samples instead of relying on KDEs and the assumptions that come with them.

However, if you cant get more samples, then you can remove the cutting yours seeing by increasing the number of bins as per the notebook I attached.

But please get more samples so you've got more confidence in your surfaces.