This PR is the last step needed to phase out nanoAOD-tools. Instead of adding numerous branches for storing corrected jets and MET with nanoAOD-tools, we would read vanilla NanoAOD directly and apply the necessary corrections on-the-fly. It is faster and more flexible compared to the equivalent task done by nanoAOD-tools, plus it also eliminates the need to store intermediate NanoAOD files and thereby reduces the human time spent on managing the said files.
The only downside is that the runtime approximately doubles to ~1k~ 100 events / second. (Although I have to point out that my previous benchmarks prior to this PR did not save the event weight branches nor split JER uncertainties, so the real increase in runtime is less than double.) This is not a huge deal because the total runtime of processing multiple years' worth of datasets still remains reasonable. If we have, say, 20B events to process, it would take about ~230~ 2300 CPU-days to get the job done (per channel). The runtime could be reduced by considering fewer systematic uncertainties when producing Ntuples for other purposes than datacard production (such as auxiliary measurement or training DNNs/BDTs). The memory consumption stays at around 1 GB (well below the designated limit of 2 GB) and stable. Below is a plot that shows the memory consumption for processing 400k NanoAODv7 HH events. The last "jump" is caused by writing the output file (it's present even when processing 1k events).
Since this PR is massive, I'll try to summarize the major points:
apply JME corrections:
AK4 jets: recalibrate, apply energy smearing, shift by JES and JER uncertainties;
AK8 jets: recalibrate, apply energy smearing, apply PUPPI SD corrections (requires SubGenJetAK8 collection), apply SD and regular mass smearing (requires GenJetAK8), apply jet mass scale corrections, shift each by relevant uncertainty;
MET: obtained by propagating fully corrected AK4 jets minus L1-calibrated AK4 jets to raw MET. In addition to the regular AK4 jets (Jet), the changes in low-pT jets that didn't pass the pT threshold in NanoAOD production (CorrT1METJet) are also propagated to the MET;
each step is configurable and carried out in JMECorrector class, which is placed under Readers;
load PU jet ID efficiency and mis-tagging SF from JMAR JSON files with correctionlib;
move PU weight application from EventInfo class to Data_to_MC_CorrectionInterface_Base;
replace disable_ak8_corr parameter with fatJet_corrections, thus forcing the user to explicitly specify the list of corrections on AK8 jets;
read NanoAODv7 by default;
write the actual jet mass instead of the mass of AK8 subjet pair (which is basically the same as SD mass that's was already written to the output file);
when switching input files, reset the list of branches that are considered already bound.
Even though the implementation is very similar to that of nanoAOD-tools, there are still some deliberate differences:
the recipe for updating MET in 2017 is very complicated, confusing and potentially flawed in nanoAOD-tools. Some of these points were also highlighted by Dennis on HN. Given that we don't plan to run this FW on Run 2 data anyways, I found that it's better to not implement it at all if it's going to be eventually scrapped;
I created the JSON files for JES and JER from the same files that we used in Run 2 analysis, but took the JMAR corrections straight from the JSON POG repository;
JMR and PUPPI SD corrections are currently hardcoded because they were missing from the JMAR JSON file;
reseed RNG for every event and jet. The seed is constructed from run-lumi-event numbers and the jet eta. In nanoAOD-tools and in CMSSW, it's constructed from the eta of the 1st jet in the collection. This point may become irrelevant in the future once the JetMET group provides dedicated SF for stochastic smearing as well.
Here are some open points:
it's not entirely obvious in which order each correction should be applied. In nanoAOD-tools, the jets are first calibrated and then smeared. The smeared jet pT is used to evaluate JES uncertainties. I'm not sure if it's the correct procedure, but that's how I've currently implemented it;
how to proceed when the input parameters to correctionlib go beyond the expected range: should we clamp the input parameters so that they fit in the range (like I've done now for JMS), or should we skip the correction altogether?
should we always revert to raw SD mass or only when applying PUPPI SD corrections (which I'm doing at the moment)?
is JMR even applicable to regular jet mass? Should it be correlated with JMR on SD mass? For now, I've assumed that it's the case for both since it's also done in nanoAOD-tools.
I think these issues can be resolved on a different time scale, and do not necessarily need to be addressed in this PR.
This PR is the last step needed to phase out nanoAOD-tools. Instead of adding numerous branches for storing corrected jets and MET with nanoAOD-tools, we would read vanilla NanoAOD directly and apply the necessary corrections on-the-fly. It is faster and more flexible compared to the equivalent task done by nanoAOD-tools, plus it also eliminates the need to store intermediate NanoAOD files and thereby reduces the human time spent on managing the said files.
The only downside is that the runtime approximately doubles to ~1k~ 100 events / second. (Although I have to point out that my previous benchmarks prior to this PR did not save the event weight branches nor split JER uncertainties, so the real increase in runtime is less than double.) This is not a huge deal because the total runtime of processing multiple years' worth of datasets still remains reasonable. If we have, say, 20B events to process, it would take about ~230~ 2300 CPU-days to get the job done (per channel). The runtime could be reduced by considering fewer systematic uncertainties when producing Ntuples for other purposes than datacard production (such as auxiliary measurement or training DNNs/BDTs). The memory consumption stays at around 1 GB (well below the designated limit of 2 GB) and stable. Below is a plot that shows the memory consumption for processing 400k NanoAODv7 HH events. The last "jump" is caused by writing the output file (it's present even when processing 1k events).
Since this PR is massive, I'll try to summarize the major points:
SubGenJetAK8
collection), apply SD and regular mass smearing (requiresGenJetAK8
), apply jet mass scale corrections, shift each by relevant uncertainty;Jet
), the changes in low-pT jets that didn't pass the pT threshold in NanoAOD production (CorrT1METJet
) are also propagated to the MET;JMECorrector
class, which is placed underReaders
;EventInfo
class toData_to_MC_CorrectionInterface_Base
;disable_ak8_corr
parameter withfatJet_corrections
, thus forcing the user to explicitly specify the list of corrections on AK8 jets;Even though the implementation is very similar to that of nanoAOD-tools, there are still some deliberate differences:
Here are some open points:
I think these issues can be resolved on a different time scale, and do not necessarily need to be addressed in this PR.