Backport 2017/18 to 2016

[bortigno]

This issue tracks the back porting of Ntupliser features from 2017 and 2018 to master_2016_94X branch.

[bortigno]

Working on branch issue-73-backport-2017-18-to-2016 which was branched out from master_2017_94X.

bortigno@lxplus629:~/workspace/x2mm16_9413/src/Ntupliser/DiMuons

[bortigno]

For the moment I followed the instruction of the README of master_2017_94X and everything compiles. Now I need to understand if I can run on 2016 94X samples, so I need a list of samples. It seems that there are a couple of samples to use as a test on python/Samples_2016_94X_v2.py, though the analyser for 2016 are outdated. I will try to use the 2017 ones first but for sure I need to update them.

Running the 2017 cfg on a 94X MC gives me this error:

cmsRun test/UFDiMuonAnalyzer_2018_02_07.py

Begin processing the 1st record. Run 1, Event 68431188, LumiSection 272717 at 27-May-2019 18:43:26.165 CEST
----- Begin Fatal Exception 27-May-2019 18:43:30 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 272717 event: 68431188 stream: 0
   [1] Running path 'p'
   [2] Calling method for module UFDiMuonsAnalyzer/'dimuons'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: double
Looking for module label: prefiringweight
Looking for productInstanceName: nonPrefiringProb

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------
27-May-2019 18:43:30 CEST  Closed file root://eoscms.cern.ch//eos/cms/store/user/abrinke1/HiggsToMuMu/samples/DYJetsToLL_M-50_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/94X_mcRun2_asymptotic_v3/2ECA84BD-B0CA-E811-9449-1866DA890B94.root
Total Number of Events Read: 1

This error might be related to issue #50 . Will investigate tomorrow.

---

Temporary to do list for 2016

I am starting a todo list here, and create issues from this later:

• [x] Update cff file. I am starting to test with 2017, but then need to update for 2016.

[bortigno]

78 and #80 are making this issue moving forward.

[bortigno]

81 includes the latest updates and back ports.

Once is merged to master_2016_94X I will start testing a pre-production, probably something like pre-prod-v16.0.0.

[bortigno]

In bortigno@lxplus615:~/workspace/x2mm16_9413/src/Ntupliser/DiMuons and master_2016_94X

git tag -m "First pre-production 2016 v3" pre-prod-v16.0.0

ok, first small bug. Fixing it and retagging it and testing a job in data and one in MC.

git tag -m "First pre-production 2016 v3" pre-prod-v16.0.1
crab submit -c ./crab_2019_06_24_18_43-pre-prod-v16.0.1/configs/SingleMu_2016B.py --dryrun

There is a problem with lumiMask

config.Data.lumiMask = '/afs/cern.ch/work/b/bortigno/x2mm16_9413/src/Ntupliser/DiMuons/../['data/JSON/2016/Cert_271036-284044_13TeV_23Sep2016ReReco_Collisions16_JSON.txt']'

Backporting now the improvement from 2018:

git checkout master_2018_102X -- crab/make_crab_script.py
vi crab/make_crab_script.py  # adapting to 2016
git commit -m "backporting improvements from master_2018_102X" crab/make_crab_script.py
[master_2016_94X 0f590d7] backporting improvements from master_2018_102X
git tag -m "First pre-production 2016 v3" pre-prod-v16.0.2
crab submit -c crab_2019_06_24_18_56-pre-prod-v16.0.2/configs/SingleMu_2016B.py --dryrun
# some problems with the json format again...
git commit -m "Changing JSON from list to string to be consistent with 2018. No need of a list." python/Samples.py
[master_2016_94X e2c5024] Changing JSON from list to string to be consistent with 2018. No need of a list.
git tag -m "First pre-production 2016 v3" pre-prod-v16.0.3
crab submit -c crab_2019_06_24_18_59-pre-prod-v16.0.3/configs/SingleMu_2016B.py --dryrun
    ERROR: AttributeError: 'Process' object has no attribute 'egamma_step'

git commit -m "#HOTFIX. Some residual from old code." crab/templates/EDAnalyzer.py
[master_2016_94X 30fd848] #HOTFIX. Some residual from old code.
git tag -m "First pre-production 2016 v3" pre-prod-v16.0.4
crab submit -c crab_2019_06_24_19_03-pre-prod-v16.0.4/configs/SingleMu_2016B.py --dryrun
#still  problems in the config 
      Invalid CMSSW configuration: 'process' object is missing attribute 'source' or the attribute is wrongly defined.

git commit -m "Fixing and polishing the cfg template for crab" crab/templates/EDAnalyzer.py
[master_2016_94X adf79e9] Fixing and polishing the cfg template for crab
git tag -m "First pre-production 2016 v3" pre-prod-v16.0.5
python crab/make_crab_script.py 
crab submit -c crab_2019_06_24_19_21-pre-prod-v16.0.5/configs/SingleMu_2016B.py --dryrun

I have a problem with the Muon json file, but I had it already in https://github.com/UFLX2MuMu/Ntupliser/issues/66 so I fixed it in the same way with 29b44c0 and e6f6107

git commit -m "adding latest 2016 SF" RunBCDEF_SF_ID.json RunBCDEF_SF_ISO.json RunGH_SF_ID.json RunGH_SF_ISO.json ../MuonTrig/EfficienciesAndSF_RunBtoF.root ../MuonTrig/EfficienciesAndSF_RunGtoH.root
git commit -m "updating path to be compatible with crab" python/Analyzer_cff.py python/Analyzer_MC_cff.py
[master_2016_94X e6f6107] updating path to be compatible with crab

now recompile test and run.

There is some more small bug fixing typos, etc. to be done. Now at 70c5c5b testing. All good locally. Trying to test on crab.

git tag -m "First pre-production 2016 v3" pre-prod-v16.0.6
python crab/make_crab_script.py 
crab submit -c crab_2019_06_24_21_11-pre-prod-v16.0.6/configs/SingleMu_2016B.py --dryrun
# I didn't update the muon SF names on Analyser_cff.py (I did only in the Analyser_MC_cff.py)

vi python/Analyzer_cff.py
 git commit --amend --no-edit
[master_2016_94X aaf92ef] Fixing muon SF path

git tag -m "First pre-production 2016 v3" pre-prod-v16.0.7
python crab/make_crab_script.py 
crab submit -c crab_2019_06_25_08_53-pre-prod-v16.0.7/configs/SingleMu_2016B.py --dryrun
# testing also the MC
crab submit -c crab_2019_06_25_08_53-pre-prod-v16.0.7/configs/H2Mu_WH_neg_125.py --dryrun

This is working. Launching the production.

One problem:

Will use CRAB configuration file crab_2019_06_25_09_05-pre-prod-v16.0.7.skim3l/configs/tZq+herwig.py
Importing CMSSW configuration crab_2019_06_25_09_05-pre-prod-v16.0.7.skim3l/analyzers/tZq+herwig.py
Finished importing CMSSW configuration crab_2019_06_25_09_05-pre-prod-v16.0.7.skim3l/analyzers/tZq+herwig.py
Sending the request to the server
The server answered with an error.
Server answered with: Invalid input parameter
Reason is: Incorrect 'General.requestName' parameter
Error Id: 6cb3b6944ea3fb1bcd19c8b28f6bfb8f
Log file is /afs/cern.ch/work/b/bortigno/x2mm16_9413/src/Ntupliser/DiMuons/logs/crab_tZq+herwig_2019_06_25_09_05_prod2016_pre-prod-v16p0p7pskim3l/crab.log

[bortigno]

Fixed a while ago. 2016 production is done.