ValidateDatacards.py Problem: Systematic type autoMCStats not supported

[okarache]

Dear experts,

could you help with the ValidateDatacards.py problem, please?

Simple datacard is used (shown under [*]) and all other steps in the card validation and pull distributions were produced with combine tools without issues using this card. But ValidateDatacards.py datacard.txt fails with full error message printed below:

[SetFlag] Changing value of flag "check-negative-bins-on-import" from 1 to 0 [SetFlag] Changing value of flag "workspaces-use-clone" from 0 to 1 Traceback (most recent call last): File "/cms/multilepton-3/olena/CMSSW_10_2_13/bin/slc7_amd64_gcc700/ValidateDatacards.py", line 110, in cb.ParseDatacard(args.cards,"","",mass=args.mass) File "/cms/multilepton-3/olena/CMSSW_10_2_13/python/CombineHarvester/CombineTools/ch.py", line 21, in ParseDatacard return self.ParseDatacard(card, analysis, era, channel, bin_id, mass) RuntimeError:

---

Context: Function ch::CombineHarvester::ParseDatacard at /cms/multilepton-3/olena/CMSSW_10_2_13/src/CombineHarvester/CombineTools/src/CombineHarvester_Datacards.cc:454 Problem: Systematic type autoMCStats not supported

---

Please report issues at https://github.com/cms-analysis/CombineHarvester/issues

---

[*] Datacard (simplified) :

imax 1 number of bins
jmax 2 number of processes minus 1
kmax 41 number of nuisance parameters
----------------------------------------------------------------------------------------------------------------------------------
shapes *     ch1  shapes_WPhiPS50mu_3L_LS0_Run2.root WPhimmMinOSSF3Lbin5B0_$PROCESS WPhimmMinOSSF3Lbin5B0_$PROCESS_$SYSTEMATIC
----------------------------------------------------------------------------------------------------------------------------------
bin          ch1
observation  1450
----------------------------------------------------------------------------------------------------------------------------------
bin          ch1   ch1   ch1
process      WPhiPS50mu   Prompt   MisID
process      0   1   2
rate         291.7487655130141   903.5581679344177   576.2285528182983
----------------------------------------------------------------------------------------------------------------------------------
BTagY16    shapeN2      -       1.0         -
BTagY17    shapeN2      -       1.0         -
BTagY18    shapeN2      -       1.0         -
LimiY16    shapeN2    1.0       1.0         -
LimiY16Y17Y18    shapeN2    1.0       1.0         -
LimiY17    shapeN2    1.0       1.0         -
LimiY17Y18    shapeN2    1.0       1.0         -
LimiY18    shapeN2    1.0       1.0         -
POGIDEleY16Y17Y18    shapeN2    1.0       1.0         -
POGIDMuY16Y17Y18    shapeN2    1.0       1.0         -
POGRecoEleY16Y17Y18    shapeN2    1.0       1.0         -
PU    shapeN2    1.0       1.0         -
ch1 autoMCStats 0 0 1

If I change the last line to ch1 autoMCStats 001 (without spaces), there is this error message is does not appear. The output is:

[SetFlag] Changing value of flag "check-negative-bins-on-import" from 1 to 0
[SetFlag] Changing value of flag "workspaces-use-clone" from 0 to 1
================================
=======Validation results=======
================================
>>>There were no warnings

Could you advise if I'm using the wrong format for autoMCStats or what is the reason for ValidateDatacards.py not recognizing ch1 autoMCStats 0 0 1 (other combine tools did not have errors regarding this line), but recognizing ch1 autoMCStats 001.

Thank you very much in advance, Olena

[adewit]

Hi @okarache, Can you share the file shapes_WPhiPS50mu_3L_LS0_Run2.root so we can try to reproduce your problem? I've never seen this before. In parallel, can you paste the commit hash printed at the top when you do git log in the CMSSW_10_2_13/src/CombineHarvester

[okarache]

Hi, @adewit ,

thank you for you response. I uploaded the shapes_WPhiPS50mu_3L_LS0_Run2.root to cernbox here: https://cernbox.cern.ch/index.php/s/Q5nPehbbMEKPz6O

Here is git log printout: /cms/multilepton-3/olena/CMSSW_10_2_13/src/CombineHarvester$ git log commit 0675d6ae83f694d836df1003b62f9919f1ca4556 (HEAD -> master, origin/master) Merge: 3efe0f2 5f16a9c Author: Andrew Gilbert andrew.gilbert@cern.ch Date: Wed Jul 21 17:06:29 2021 +0200

Merge pull request #260 from threiten/dev_skipInitialFitImpacts

Added option to skip running initial fit for impact plots.

Thank you and with best wishes, Olena

[adewit]

Hi @okarache, thanks for reporting this - it's an interplay between the number of processes you have in the datacard, which means that the normal systematics lines are exactly as long (in terms of number of items) as the full autoMCStats line, leading to the wrong parsing loop being invoked. When you removed the spaces between the numbers, they are interpreted as a single item and therefore the line length is different than the normal systematics line, which meant the parsing did work. This behaviour is of course not intended. Since the second and third numeric argument passed in the datacard for autoMCStats default to 0 and 1, you can run the validation tool with just ch1 autoMCStats 0, or you can pick up the change from #274, which we can hopefully merge soon.

[okarache]

Thank you, @adewit !