Open rmnmllr opened 1 year ago
Hi, the short answer is that yes, this ultimately is a problem with the input. cabinetry
doesn't provide any functionality to automatically do something to work around this, because there is not really a good solution here. There just are not enough events to properly estimate the distribution of this template.
Things you could try out:
Not applicable in this specific case where the total sum is negative, but other things that can help if you have a negative bin:
"Smoothing": {"Algorithm": "353QH, twice"}
,Thank you for you quick answer and suggestions, we will discuss internally on how to continue!
For the smoothing, can this be applied for all systematics with one setting in the config or do I have to do:
- Name: "EG_RESOLUTION_ALL"
Up:
VariationPath: "EG_RESOLUTION_ALL__1up"
Down:
VariationPath: "EG_RESOLUTION_ALL__1down"
Type: "NormPlusShape"
Samples: ["multiboson", "singletop", "Zll", "Ztautau", "Wjets", "ttbar"]
Smoothing:
Algorithm: "353QH, twice"
for each systematics in the YAML? I did a quick test with/without smoothing but I don't see any difference in the saved workspace. How does this come into play?
Currently this is a per-systematic setting, so no easy way to apply this to everything at once. You could also split such a systematic that acts on multiple samples into two pieces, allowing you to only apply smoothing where needed:
- Name: "EG_RESOLUTION_ALL"
Up:
VariationPath: "EG_RESOLUTION_ALL__1up"
Down:
VariationPath: "EG_RESOLUTION_ALL__1down"
Type: "NormPlusShape"
Samples: ["multiboson", "singletop", "Ztautau", "Wjets", "ttbar"]
- Name: "EG_RESOLUTION_ALL_with_smoothing"
ModifierName: "EG_RESOLUTION_ALL"
Up:
VariationPath: "EG_RESOLUTION_ALL__1up"
Down:
VariationPath: "EG_RESOLUTION_ALL__1down"
Type: "NormPlusShape"
Samples: ["Zll"]
Smoothing:
Algorithm: "353QH, twice"
Note the use of ModifierName
here to ensure the nuisance parameters have the same name in the workspace and will end up being correlated.
The smoothing happens during cabinetry.templates.postprocess
, are you perhaps skipping this step?
Dear @alexander-held, Unfortunately we have another problem with (most likely) negative yields.
In our analysis we have systematics with up and down variation histograms. We want to add one of these systematics, labelled
JET_Pileup_RhoTopology
, with aNormPlusShape
modifier. I define it in the config with:Now reading in the config and loading all the histogram data works fine, but when
cabinetry
reaches the plotting withcabinetry.visualize.data_mc
it aborts with:Tracing back the NaN I see already on the yield tables that the
Zll
samples have NaN for all bins. I investigated further and found that the NaN are most likely coming from theworkspace.normplusshape_modifiers
method giving back negativenorm_effect_up
andnorm_effect_down
. There must be a logarithm function somewhere then where the NaNs are produced.They are negative because the
sum(histogram_up.yields)
andsum(histogram_down.yields)
are negative due to a single bin (250 - 300 GeV) having a large negative value and dragging down the sum (see the debug print below).Now If I set
norm_effect_up = abs(norm_effect_up)
andnorm_effect_down = abs(norm_effect_down)
I get around the NaN and have no problems. But how could we solve this problem? Is it a problem of our inputs or cancabinetry
handle something to get around this issue?Unfortunately I can't provide a quick minimal example to reproduce the problem, but from these debug prints it should be clear of what's going in and where the problem appears:
Please let me know if you need anything more.
Many thanks in advance! Roman