Porting of NanoAOD-tools to CMSSW

lenzip commented 1 year ago

NanoAOD-tools is a set of tools originally developed by M. Peruzzi and collaborators for looping over NanoAOD files with PyROOT. It consists of a "framework" part handling the event loop and I/O and a set of correction modules. It is virtually unsupported since several years, but it still is a favorite among analyzers, with several hundred forks, which flourished as the "central" support decreased.

Following a discussion in December 2022, Common Analysis Tools (CAT, @lenzip and @clelange are the L2s) has decided to revamp support of nanoAOD-tools, which is going to be supported within the CAT-DPROC L3 area (@andrzejnovak and @ttedesch are the L3s). The way forward was further discussed in this talk, with @namapane proposing to port to CMSSW, under a new package with proposed name PhysicsTools/NanoAOD-tools, the "framework" part, which is very stable, and keep in a separate repository the correction part (this will be handled by CAT in a UserCode-type repository). A prototype here is already available.

We would like to proceed to include this new package in CMSSW in the current release cycle, and then backport to the relevant releases, in particular 10_6_X for NanoAODv9 and 12_6_X for NanoAODv11.

I am tagging also the XPOG conveners @vlimant and @simonepigazzini for their information

cmsbuild commented 1 year ago

A new Issue was created by @lenzip .

@Dr15Jones, @perrotta, @dpiparo, @rappoccio, @makortel, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

makortel commented 1 year ago

The way forward was further discussed in this talk, with @namapane proposing to port to CMSSW, under a new package with proposed name PhysicsTools/NanoAOD-tools

Do I understand correctly from the talk and the ptototype that the proposed package name is actually PhysicsTools/NanoAODTools?

revamp support of nanoAOD-tools, which is going to be supported within the CAT-DPROC L3 area

Who would take the maintenance ownership (i.e. reviews PRs, be first responder for any problems) of the new package? XPOG? CAT-DPROC L3?

Are the users expected to use this code within CMSSW developer area (i.e. starting with cmsrel), or as a "standalone" package that just happens to be part of CMSSW (i.e. starting with git clone)?

makortel commented 1 year ago

assign xpog, orp

(I'm not fully sure who exactly to assign to)

cmsbuild commented 1 year ago

New categories assigned: orp,xpog

@dpiparo,@rappoccio,@perrotta,@simonepigazzini,@vlimant you have been requested to review this Pull request/Issue and eventually sign? Thanks

lenzip commented 1 year ago

On 01/06/23 16:09, Matti Kortelainen wrote:

The way forward was further discussed in this talk
<https://indico.cern.ch/event/1277996/#2-nanoaod-tools-a-way-forward>,
with @namapane <https://github.com/namapane> proposing to port to
CMSSW, under a new package with proposed name
PhysicsTools/NanoAOD-tools
Do I understand correctly from the talk and the ptototype https://github.com/namapane/cmssw/tree/NanoAODTools-dev/PhysicsTools/NanoAODTools that the proposed package name is actually |PhysicsTools/NanoAODTools|?

Yes, sorry.
revamp support of nanoAOD-tools, which is going to be supported
within the CAT-DPROC L3 area
Who would take the maintenance ownership (i.e. reviews PRs, be first responder for any problems) of the new package? XPOG? CAT-DPROC L3?

CAT-DPROC L3s

Are the users expected to use this code within CMSSW developer area (i.e. starting with |cmsrel|), or as a "standalone" package that just happens to be part of CMSSW (i.e. starting with |git clone|)?

Mainly within CMSSW, but we will try to make standalone use possible, as it used to be before. Most users used to use it in CMSSW anyways.

— Reply to this email directly, view it on GitHub https://github.com/cms-sw/cmssw/issues/41839#issuecomment-1572130138, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFOAVQUE6C6UGEH4V4MJE3XJCPBPANCNFSM6AAAAAAYWURFYQ. You are receiving this because you were mentioned.Message ID: @.***>

makortel commented 1 year ago

revamp support of nanoAOD-tools, which is going to be supported within the CAT-DPROC L3 area

Who would take the maintenance ownership (i.e. reviews PRs, be first responder for any problems) of the new package? XPOG? CAT-DPROC L3?

CAT-DPROC L3s

I guess this would mean we'd have to create a new signatory category (categories.py, categories_map.py) and GitHub team (https://github.com/orgs/cms-sw/teams). (I'd be tempted to suggest to reuse the old, de-facto abandoned analysis-l2, but probably that has too much of history that a new entity would be better?)

simonepigazzini commented 1 year ago

as xpog we support the idea of a new signature category.

lenzip commented 1 year ago

Hello, I just wanted to know if there is anything to do from our side to move this forward.

namapane commented 1 year ago

Hi @lenzip, I can make a PR as soon as I find some time to fix the standalone setup scripts and doc (after the CMS week and an exams session...)

namapane commented 1 year ago

Hi all, I just made a draft PR #42323 with my proposed modifications. It took some time mostly to fix examples and documentation...

Some notes:

Intended mainly to provide the postprocessor for skimming etc. in a CMSSW environment, however as was requested I also provided a standalone chekout script.
I kept the original crab example scripts in, but if deemed inappropriate I think they can be removed from the package.
Example correction modules based on correctionlib should go in a separate package outside cmssw, as per previous discussions.

Comments are welcome.

lenzip commented 1 year ago

Sorry I got lost in the signature discussion. Is there anything missing for this PR?