Closed andreab1997 closed 1 year ago
Note that for
ATLAS_WP_7TEV
the two runs are digit by digit the same and good and this is re-computed, not convertedATLAS_WP_JET_8TEV_PT-atlas-atlas-wjets-arxiv-1711.03296-xsec002
the thing went wrong as @andreab1997 said and it is convertedATLASWZRAP36PB-ATLAS-arXiv:1109.5141-Wminus_eta3
both versions are fine, but not the same digit by digit and it is convertedso the conjecture by @cschwan seems correct there is something fishy with the APPL conversion; however, keep in mind the grids were (most likely) not re-converted, but they were converted long ago
Good observation. I took @andreab1997's two lists and diff'ed them, which leads to the following two lists; the grids that seem to be unaffected are:
ATLAS_DY_7TEV_49FB_HIMASS.pineappl.lz4
ATLAS_DY_7TEV_CC.pineappl.lz4
ATLAS_DY_7TEV_CF.pineappl.lz4
ATLAS_DY_7TEV_LOMASS_EXT.pineappl.lz4
ATLAS_TTBARTOT_13TEV_FULLLUMI-TOPDIFF13TEVTOT.pineappl.lz4
ATLASTTBARTOT7TEV-TOPDIFF7TEVTOT.pineappl.lz4
ATLASTTBARTOT8TEV-TOPDIFF8TEVTOT.pineappl.lz4
ATLAS_WM_7TEV.pineappl.lz4
ATLAS_WP_7TEV.pineappl.lz4
CMS_DY_7TEV_2D.pineappl.lz4
CMSTTBARTOT13TEV-TOPDIFF13TEVTOT.pineappl.lz4
CMSTTBARTOT7TEV-TOPDIFF7TEVTOT.pineappl.lz4
CMSTTBARTOT8TEV-TOPDIFF8TEVTOT.pineappl.lz4
CMSWEASY840PB-CMS-PAS-SMP-12-001-Wminus_eta3.pineappl.lz4
D0ZRAP-D0_ZNORM_MCgrid_500M.pineappl.lz4
DYE605.pineappl.lz4
DYE866P.pineappl.lz4
DYE866R_D.pineappl.lz4
DYE866R_P.pineappl.lz4
DYE906R_D_bin_0.pineappl.lz4
DYE906R_D_bin_1.pineappl.lz4
DYE906R_D_bin_2.pineappl.lz4
DYE906R_D_bin_3.pineappl.lz4
DYE906R_D_bin_4.pineappl.lz4
DYE906R_D_bin_5.pineappl.lz4
DYE906R_D_bin_6.pineappl.lz4
DYE906R_D_bin_7.pineappl.lz4
DYE906R_D_bin_8.pineappl.lz4
DYE906R_D_bin_9.pineappl.lz4
DYE906R_P_bin_0.pineappl.lz4
DYE906R_P_bin_1.pineappl.lz4
DYE906R_P_bin_2.pineappl.lz4
DYE906R_P_bin_3.pineappl.lz4
DYE906R_P_bin_4.pineappl.lz4
DYE906R_P_bin_5.pineappl.lz4
DYE906R_P_bin_6.pineappl.lz4
DYE906R_P_bin_7.pineappl.lz4
DYE906R_P_bin_8.pineappl.lz4
DYE906R_P_bin_9.pineappl.lz4
LHCB_DY_13TEV_DIELECTRON.pineappl.lz4
LHCB_DY_13TEV_DIMUON.pineappl.lz4
LHCB_DY_7TEV_940PB.pineappl.lz4
LHCB_DY_7TEV.pineappl.lz4
LHCB_DY_8TEV_2FB.pineappl.lz4
LHCB_DY_8TEV.pineappl.lz4
LHCB_WM_7TEV.pineappl.lz4
LHCB_WM_8TEV.pineappl.lz4
LHCB_WP_7TEV.pineappl.lz4
LHCB_WP_8TEV.pineappl.lz4
and the list of clearly affected grids:
ATLASDY2D8TEV-aMCfast_obs_0.pineappl.lz4
ATLASDY2D8TEV-aMCfast_obs_1.pineappl.lz4
ATLASDY2D8TEV-aMCfast_obs_2.pineappl.lz4
ATLASDY2D8TEV-aMCfast_obs_3.pineappl.lz4
ATLASDY2D8TEV-aMCfast_obs_4.pineappl.lz4
ATLAS_DY_2D_8TEV_LOWMASS-ATLAS_DY_2D_8TEV-BIN1_rapidity.pineappl.lz4
ATLAS_DY_2D_8TEV_LOWMASS-ATLAS_DY_2D_8TEV-BIN2_rapidity.pineappl.lz4
ATLAS_DY_2D_8TEV_LOWMASS-ATLAS_DY_2D_8TEV-BIN3_rapidity.pineappl.lz4
ATLAS_DY_2D_8TEV_LOWMASS-ATLAS_DY_2D_8TEV-BIN4_rapidity.pineappl.lz4
ATLAS_DY_2D_8TEV_LOWMASS-ATLAS_DY_2D_8TEV-BIN5_rapidity.pineappl.lz4
ATLAS_DY_2D_8TEV_LOWMASS-ATLAS_DY_2D_8TEV-BIN6_rapidity.pineappl.lz4
ATLAS_DY_2D_8TEV_LOWMASS-ATLAS_DY_2D_8TEV-BIN7_rapidity.pineappl.lz4
ATLASPHT15-ATLASPHT15_Et_1bin.pineappl.lz4
ATLASPHT15-ATLASPHT15_Et_2bin.pineappl.lz4
ATLASPHT15-ATLASPHT15_Et_3bin.pineappl.lz4
ATLASPHT15-ATLASPHT15_Et_4bin.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_7TEV_TBAR_RAP_NORM-aMCfast_obs_0.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_7TEV_TBAR_RAP_NORM-aMCfast_obs_1.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_7TEV_TBAR_RAP_NORM-aMCfast_obs_2.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_7TEV_TBAR_RAP_NORM-ATLAS_SINGLETOP_TCH_R_7TEV_TB.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_7TEV_T_RAP_NORM-aMCfast_obs_0.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_7TEV_T_RAP_NORM-aMCfast_obs_1.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_7TEV_T_RAP_NORM-aMCfast_obs_2.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_7TEV_T_RAP_NORM-ATLAS_SINGLETOP_TCH_R_7TEV_T.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_8TEV_TBAR_RAP_NORM-aMCfast_obs_0.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_8TEV_TBAR_RAP_NORM-aMCfast_obs_1.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_8TEV_TBAR_RAP_NORM-aMCfast_obs_2.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_8TEV_TBAR_RAP_NORM-ATLAS_SINGLETOP_TCH_R_8TEV_TB.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_8TEV_T_RAP_NORM-aMCfast_obs_0.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_8TEV_T_RAP_NORM-aMCfast_obs_1.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_8TEV_T_RAP_NORM-aMCfast_obs_2.pineappl.lz4
ATLAS_SINGLETOP_TCH_DIFF_8TEV_T_RAP_NORM-ATLAS_SINGLETOP_TCH_R_8TEV_T.pineappl.lz4
ATLAS_SINGLETOP_TCH_R_13TEV-ATLAS_SINGLETOP_TCH_R_13TEV_TB.pineappl.lz4
ATLAS_SINGLETOP_TCH_R_13TEV-ATLAS_SINGLETOP_TCH_R_13TEV_T.pineappl.lz4
ATLAS_SINGLETOP_TCH_R_7TEV-ATLAS_SINGLETOP_TCH_R_7TEV_TB.pineappl.lz4
ATLAS_SINGLETOP_TCH_R_7TEV-ATLAS_SINGLETOP_TCH_R_7TEV_T.pineappl.lz4
ATLAS_TTB_8TEV_2L_TTRAP.pineappl.lz4
ATLAS_TTB_8TEV_LJ_TRAP.pineappl.lz4
ATLAS_TTB_8TEV_LJ_TRAP_TOT.pineappl.lz4
ATLAS_TTB_8TEV_LJ_TTRAP.pineappl.lz4
ATLAS_TTB_8TEV_LJ_TTRAP_TOT.pineappl.lz4
ATLAS_WM_JET_8TEV_PT-atlas-atlas-wjets-arxiv-1711.03296-xsec003.pineappl.lz4
ATLAS_WP_JET_8TEV_PT-atlas-atlas-wjets-arxiv-1711.03296-xsec002.pineappl.lz4
ATLASWZRAP36PB-ATLAS-arXiv:1109.5141-Wminus_eta3.pineappl.lz4
ATLASWZRAP36PB-ATLAS-arXiv:1109.5141-Wplus_eta4.pineappl.lz4
ATLASWZRAP36PB-ATLAS-arXiv:1109.5141-Z0_eta34.pineappl.lz4
ATLAS_WZ_TOT_13TEV-ATLASWZTOT13TEV81PB_WM_tot.pineappl.lz4
ATLAS_WZ_TOT_13TEV-ATLASWZTOT13TEV81PB_WP_tot.pineappl.lz4
ATLAS_WZ_TOT_13TEV-ATLASWZTOT13TEV81PB_Z_tot.pineappl.lz4
ATLASZPT8TEVMDIST-ATLASZPT8TEV-MLLBIN1_ptZ.pineappl.lz4
ATLASZPT8TEVMDIST-ATLASZPT8TEV-MLLBIN2_ptZ.pineappl.lz4
ATLASZPT8TEVMDIST-ATLASZPT8TEV-MLLBIN3_ptZ.pineappl.lz4
ATLASZPT8TEVMDIST-ATLASZPT8TEV-MLLBIN4_ptZ.pineappl.lz4
ATLASZPT8TEVMDIST-ATLASZPT8TEV-MLLBIN6_ptZ.pineappl.lz4
ATLASZPT8TEVYDIST-ATLASZPT8TEVYDIST-BIN1_ptZ.pineappl.lz4
ATLASZPT8TEVYDIST-ATLASZPT8TEVYDIST-BIN2_ptZ.pineappl.lz4
ATLASZPT8TEVYDIST-ATLASZPT8TEVYDIST-BIN3_ptZ.pineappl.lz4
ATLASZPT8TEVYDIST-ATLASZPT8TEVYDIST-BIN4_ptZ.pineappl.lz4
ATLASZPT8TEVYDIST-ATLASZPT8TEVYDIST-BIN5_ptZ.pineappl.lz4
ATLASZPT8TEVYDIST-ATLASZPT8TEVYDIST-BIN6_ptZ.pineappl.lz4
CDFZRAP_NEW-CDF_ZRAP_MCgrid_500M_last_two_bins_combined.pineappl.lz4
CMS_SINGLETOP_TCH_R_13TEV-CMS_SINGLETOP_TCH_R_13TEV_TB.pineappl.lz4
CMS_SINGLETOP_TCH_R_13TEV-CMS_SINGLETOP_TCH_R_13TEV_T.pineappl.lz4
CMS_SINGLETOP_TCH_R_8TEV-CMS_SINGLETOP_TCH_R_8TEV_TB.pineappl.lz4
CMS_SINGLETOP_TCH_R_8TEV-CMS_SINGLETOP_TCH_R_8TEV_T.pineappl.lz4
CMS_SINGLETOP_TCH_TOT_7TEV-CMS_SINGLETOP_TCH_R_7TEV_TB.pineappl.lz4
CMS_SINGLETOP_TCH_TOT_7TEV-CMS_SINGLETOP_TCH_R_7TEV_T.pineappl.lz4
CMSTOPDIFF8TEVTTRAPNORM-TOPDIFF8TEVTTRAP.pineappl.lz4
CMS_TTB_13TEV_2L_TRAP.pineappl.lz4
CMS_TTB_13TEV_LJ_TRAP.pineappl.lz4
CMS_TTB_5TEV_TOT.pineappl.lz4
CMS_TTB_8TEV_2D_TTM_TRAP.pineappl.lz4
CMS_TTB_8TEV_2D_TTM_TRAP_TOT.pineappl.lz4
CMSWEASY840PB-CMS-PAS-SMP-12-001-Wminus_eta3.pineappl.lz4
CMSWEASY840PB-CMS-PAS-SMP-12-001-Wplus_eta4.pineappl.lz4
CMSWMASY47FB-CMSWMASY47FB-Wm-eta3.pineappl.lz4
CMSWMASY47FB-CMSWMASY47FB-Wp-eta4.pineappl.lz4
CMSWMU8TEV-CMSWMU8TEV_WM_leptrap.pineappl.lz4
CMSWMU8TEV-CMSWMU8TEV_WP_leptrap.pineappl.lz4
CMSZDIFF12-CMSZDIFF12-BIN1_ptZ.pineappl.lz4
CMSZDIFF12-CMSZDIFF12-BIN2_ptZ.pineappl.lz4
CMSZDIFF12-CMSZDIFF12-BIN3_ptZ.pineappl.lz4
CMSZDIFF12-CMSZDIFF12-BIN4_ptZ.pineappl.lz4
CMSZDIFF12-CMSZDIFF12-BIN5_ptZ.pineappl.lz4
D0WMASY-grid-40-6-15-3-Wminus_wly_pt25.pineappl.lz4
D0WMASY-grid-40-6-15-3-Wplus_wly_pt25.pineappl.lz4
D0ZRAP-D0_ZRAP_MCgrid_500M.pineappl.lz4
@andreab1997 can you check whether this is a NNLO problem? i.e can you compute the NLO EKO counterpart from the grid and compare grid vs FK at NLO?
using an old version of ATLASWZRAP36PB-ATLAS-arXiv:1109.5141-Z0_eta34.pineappl.lz4
which I had floating around somewhere I got
$ pineappl evolve too.pineappl.lz4 t.tar tee.pineappl.lz4 NNPDF40_nlo_as_01180
LHAPDF 6.4.0 loading /home/felix/local/share/LHAPDF/NNPDF40_nlo_as_01180/NNPDF40_nlo_as_01180_0000.dat
NNPDF40_nlo_as_01180 PDF set, member #0, version 1; LHAPDF ID = 331700
b Grid FkTable rel. diff
-+-----------+-----------+-------------
0 1.3463245e5 1.3450064e5 -9.7904355e-4
1 1.3347254e5 1.3334423e5 -9.6133745e-4
with everything at NLO (note that the EKO was also an old eko so I could run pineappl evolve
)
too.pineappl.lz4
is a re-optimized version of the original grid and that seems to be more or less fine:
$ pineappl diff t.pineappl.lz4 too.pineappl.lz4 NNPDF40_nlo_as_01180 --ignore-lumis
LHAPDF 6.4.0 loading /home/felix/local/share/LHAPDF/NNPDF40_nlo_as_01180/NNPDF40_nlo_as_01180_0000.dat
NNPDF40_nlo_as_01180 PDF set, member #0, version 1; LHAPDF ID = 331700
b x1 O(as^0 a^0) O(as^1 a^0)
-+---+---+-----------+-----------+----------+-----------+-----------+----------
0 0 0.4 1.2425259e5 1.2425259e5 2.220e-16 1.0379858e4 1.0379858e4 8.882e-16
1 0.4 0.8 1.2306826e5 1.2306826e5 2.220e-16 1.0404277e4 1.0404277e4 -4.774e-15
Note the additional --ignore-lumis
though
CMS_TTB_8TEV_2D_TTM_TRAP_TOT.pineappl.lz4
shows the largest differences and this is a grid that we generated ourselves, so it's mostly the converted grids but not only them.
@andreab1997 can you check whether this is a NNLO problem? i.e can you compute the NLO EKO counterpart from the grid and compare grid vs FK at NLO?
using an old version of
ATLASWZRAP36PB-ATLAS-arXiv:1109.5141-Z0_eta34.pineappl.lz4
which I had floating around somewhere I got$ pineappl evolve too.pineappl.lz4 t.tar tee.pineappl.lz4 NNPDF40_nlo_as_01180 LHAPDF 6.4.0 loading /home/felix/local/share/LHAPDF/NNPDF40_nlo_as_01180/NNPDF40_nlo_as_01180_0000.dat NNPDF40_nlo_as_01180 PDF set, member #0, version 1; LHAPDF ID = 331700 b Grid FkTable rel. diff -+-----------+-----------+------------- 0 1.3463245e5 1.3450064e5 -9.7904355e-4 1 1.3347254e5 1.3334423e5 -9.6133745e-4
with everything at NLO (note that the EKO was also an old eko so I could run
pineappl evolve
)
too.pineappl.lz4
is a re-optimized version of the original grid and that seems to be more or less fine:$ pineappl diff t.pineappl.lz4 too.pineappl.lz4 NNPDF40_nlo_as_01180 --ignore-lumis LHAPDF 6.4.0 loading /home/felix/local/share/LHAPDF/NNPDF40_nlo_as_01180/NNPDF40_nlo_as_01180_0000.dat NNPDF40_nlo_as_01180 PDF set, member #0, version 1; LHAPDF ID = 331700 b x1 O(as^0 a^0) O(as^1 a^0) -+---+---+-----------+-----------+----------+-----------+-----------+---------- 0 0 0.4 1.2425259e5 1.2425259e5 2.220e-16 1.0379858e4 1.0379858e4 8.882e-16 1 0.4 0.8 1.2306826e5 1.2306826e5 2.220e-16 1.0404277e4 1.0404277e4 -4.774e-15
Note the additional
--ignore-lumis
though
I already did and yes, NLO seems to be fine. Let me post here the usual comparison. comparison_405.txt
Does that mean there's some interference with the NNLO K-factor to grid conversion and evolution? That's strange.
Does that mean there's some interference with the NNLO K-factor to grid conversion and evolution? That's strange.
I don't believe so because the same problem that I had with the grids with included kfactors, I also had with the usual grids once I used them at NNLO (which is what we usually do in order to use the kfactor at the end)
To effectively debug this, I need to upgrade pineappl evolve ...
for which I've opened #227.
This isn't a bug in PineAPPL.
Computing FKs with
pineappl==0.6.a5
and comparing their predictions with the predictions that can be obtained from the grids (i.e. usingpineko compare
) results in some of the FK predictions to be off by up to a 100% factor. Considering all the hadronic FKs in NNPDF40 datasets, most of them are fine but there are a (not negligible) few which are off.This is the comparison I mentioned
comparison_424.txt
Just skipping the grid optimization in
pineko
(i.e.grid.optimize()
) the problem disappears.Here is the same comparison but with the new FKs comparison_424.txt
To simplify the debug process, here is the list of the mostly affected dataset:
which of course can be inferred by the comparisons I posted.
@cschwan @AleCandido @felixhekhorn
As far as I know at the moment only me, @niclaurenti and @giacomomagni are computing fktables. I tag you here so that you are aware and you can check your FKs as well.