Split event weight sums out of ExTreeMaker

[pieterdavid]

When running on a dataset where I only consider few events (selected by a prescaled trigger), I want to apply a filter before the framework module (because the modules to redo the JEC, b-tagging etc. are run for every event that reaches that stage, which makes it much slower than needed). What prevented this was the sum of weights (on MC), which needs to see all events (it's also useful for data to know how many events were processed), so I moved this to a separate module that keeps the weight sums, and puts the information for the event in a new product (simple struct; this is needed because the ExTreeMaker creates the TTree, and this shouldn't change). Since this new producer also fills the weights to the output file, this needs to be accessible by all modules, enter the TFileService. From that comes the one backward-incompatible change: it is then easier (and more natural) to write the tree in the "framework" directory (so downstream code would need to myFile->Get("framework/t") if we don't revert that - I did change it in the tests to see if they pass). I added a trivial addPreFilter(module) method to the python Framework class for consistency, the way to add a filter is

framework = Framework(...)
framework.process.myFilter = cms.EDFilter(...)
famework.addPreFilter(framework.process.myFilter)

Testing done: processed 100000 events, all modules except genInfoEventWeights and my filter (and triggerResults) are run only 31 times while event_weight_sum is still 100000 (and the job finished in 86.736s).

[pieterdavid]

ping... could anyone have a look at these changes? They are not entirely transparent so we may need to coordinate before merging them, but they can be reviewed

[vischia]

Test fails; looks like the "if" might be needed? Perhaps the CI test uses a tree that has "t" instead of "framework/t"?

[vischia]

By the way, how do I access the full log of the integration test? I clicked on the red cross, but I can't find detailed logs

[pieterdavid]

The tests failed because they still try to get the input files from Olivier's public folder (for this branch). For the full log you can click the red cross, then "details", then "console output" on the left (or "details" at the bottom of the page in the PR status and then console output) - assuming you are authorized to access the jenkins instance...

[vischia]

Got it, and it seems I actually have auth, thanks!