Postprocessing crab job with histograms

[alefisico]

Hi all I want to run a postprocessing job that adds a variable to the current nanoAOD and creates some histograms. Running locally and in condor works perfectly, but sending them in crab I am getting the following error:

== CMSSW:     getattr( self, 'variable' ).Fill( var )
== CMSSW: AttributeError: 'resolvedAnalyzer' object has no attribute 'variable'

which basically it is not finding the histogram that I am sure I am including it in the beginJob part. Before give you more details, is this behavior expected? Can I create a job in crab that creates histogram and trees? thanks for the help!

[NJManganelli]

You can fill histograms and trees simultaneously in CRAB.

If you were using a fork of the crab submission included in the repository and a PostProcessor module, you would need to add 'hist.root' to the list in the crab configuration, in the variable including 'tree.root', so that they are transferred back to your group/user space. Otherwise, it should work like typical.

You'll probably need to post those details for anyone to say more.

[alefisico]

Hi @NJManganelli thanks for the reply. Here more details:

• Here is the Postprocessor part:

  p = PostProcessor(
  '.', (inputFiles() if not args.iFile else [args.iFile]),
  cut=cuts,
  modules=listOfModules,
  provenance=True,
  haddFileName = "nano_postprocessed.root",
  histFileName = "histograms.root",
  histDirName = 'tthbb13',
  jsonInput=runsAndLumis(),
  prefetch=True,
  longTermCache=True,
  )

• So this is my crab configuration:

  from WMCore.Configuration import Configuration
  config = Configuration()
  config.section_('General')
  config.General.workArea = 'crab_projects'
  config.General.requestName = 'ttHTobb_M125_TuneCP5_13TeV-powheg-pythia8_tthbb13_evenSimplerJob_v00_test'
  config.section_('JobType')
  config.JobType.inputFiles = ['evenSimplerJob_nanoAODPostproc.py', 'keep_and_drop.txt', 'haddnano.py', '../data/RunABCD_SF_ID.root', '../data/RunABCD_SF_ISO.root', '../data/EfficienciesAndSF_2018Data_AfterMuonHLTUpdate.root', '../data/SingleEG_JetHT_Trigger_Scale_Factors_ttHbb_Legacy2018_v1.root']
  config.JobType.scriptExe = 'runPostProcEvenSimplerJob_ttHTobb_M125_TuneCP5_13TeV-powheg-pythia8.sh'
  config.JobType.pluginName = 'Analysis'
  config.JobType.outputFiles = ['nano_postprocessed.root', 'histograms.root']
  config.JobType.sendPythonFolder = True
  config.JobType.psetName = 'PSet.py'
  config.JobType.maxJobRuntimeMin = 2750
  config.section_('Data')
  config.Data.inputDataset = '/ttHTobb_M125_TuneCP5_13TeV-powheg-pythia8/algomez-NANOAOD_v02-5157e087a222b5255c63dabe0cebaee6/USER'
  config.Data.outputDatasetTag = 'tthbb13_evenSimplerJob_v00_test'
  config.Data.unitsPerJob = 1
  config.Data.ignoreLocality = True
  config.Data.inputDBS = 'phys03'
  config.Data.splitting = 'FileBased'
  config.Data.outLFNDirBase = '/store/user/algomez/ttH/nanoAODPostproc/'
  config.section_('Site')
  config.Site.whitelist = ['T2_CH_CSCS']
  config.Site.storageSite = 'T2_CH_CSCS'

do you spot something missing? Again, the module that I am running works fine locally and in condor.

Thanks again!

[NJManganelli]

This configuration is sufficient for me to have both outputs: crab_cfg.py

from WMCore.Configuration import Configuration
from CRABClient.UserUtilities import config, getUsernameFromSiteDB

config = Configuration()

config.section_("General")
config.General.requestName = 'ttWH_2017'
config.General.transferLogs = True
config.section_("JobType")
config.JobType.pluginName = 'Analysis'
config.JobType.psetName = 'PSet.py'
config.JobType.scriptExe = 'crab_script_ttWH_2017.sh'
config.JobType.inputFiles = ['crab_script_ttWH_2017.py', 'haddnano.py',]
config.JobType.outputFiles = ['ttWH_2017_hist.root']
config.JobType.sendPythonFolder  = True
config.section_("Data")
config.Data.inputDataset = '/TTWH_TuneCP5_13TeV-madgraph-pythia8/RunIIFall17NanoAODv5-PU2017_12Apr2018_Nano1June2019_new_pmx_102X_mc2017_realistic_v7-v1/NANOAODSIM'
config.Data.inputDBS = 'global'
config.Data.splitting = 'FileBased'
config.Data.unitsPerJob = 6

config.Data.outLFNDirBase = '/store/user/%s/Stage_NANOv5_to_LogicChain/crab_20190917_1844' % (getUsernameFromSiteDB())
config.Data.outputDatasetTag = 'ttWH_2017'
config.section_("Site")
config.Site.storageSite = 'T2_CH_CERN'

PSet.py

import FWCore.ParameterSet.Config as cms
process = cms.Process('NANO')
process.source = cms.Source("PoolSource", fileNames = cms.untracked.vstring())
process.source.fileNames = [
        '../../test/lzma.root' ##you can change only this line
]
process.maxEvents = cms.untracked.PSet(input = cms.untracked.int32(10))
process.output = cms.OutputModule("PoolOutputModule", fileName = cms.untracked.string('tree.root'))
process.out = cms.EndPath(process.output)