search

cms-l1t-offline / cmssw

CMS Offline Software
cms-sw.github.io/cmssw
Apache License 2.0
16 stars 26 forks source link

HLT/L1T Interface Update #103

Open mulhearn opened 8 years ago

mulhearn commented 8 years ago

We are deprecating L1Extra for Stage-2. This requires updates to the HLT/L1T interface.

The topic branch for this issue is here:

https://github.com/cms-l1t-offline/cmssw/tree/l1t-hlt-dev-CMSSW_8_0_0_pre4

The testing recipe is:

cmsrel CMSSW_8_0_0_pre4
cd CMSSW_8_0_0_pre4/src
cmsenv
git cms-merge-topic cms-l1t-offline:l1t-hlt-dev-CMSSW_8_0_0_pre4
scram b -j 8
cmsRun L1Trigger/L1TCommon/test/devHLT.py

Currently the main changes are to

The test code runs L1T and the new HLT module, however, including the module in standard HLT sequences still leads to crashes.

mulhearn commented 8 years ago

Martin points me to the main HLT code needing an update:

https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/HLTrigger/HLTfilters/src/HLTLevel1GTSeed.cc

mulhearn commented 8 years ago

Just merged the packer/unpacker code from GT into the dev recipe branch and it seems to work. So ready to update L1Repack.

Also started updating the HLT filter which selects the seeds which fired specified L1 triggers. Decided that for pass1 the updated module will simply provide all the L1 trigger seeds without selecting based on what trigger was passed. Then, once we have the replacement for L1GtUtils working, we'll extend the HLT filter to use it. There's one additional layer of parsing that handles making arbitrary combinations of L1 trigger bits, but I think that can be recycled from the legacy system.

mulhearn commented 8 years ago

I have a new HLT filter for HLTL1TSeed to replace HLTLevel1GTSeed. At the moment it still doesn't do much except not die. This lets me see what else is downstream that depends on l1extra. The list is fortunately not as long as I feared:

#   modified:   HLTrigger/Muon/src/HLTMuonL1Filter.cc
#   modified:   RecoEgamma/EgammaHLTProducers/interface/HLTRecHitInAllL1RegionsProducer.h
#   modified:   RecoMuon/L2MuonSeedGenerator/src/L2MuonSeedGenerator.cc
#   modified:   RecoTauTag/HLTProducers/src/CaloTowerCreatorForTauHLT.cc

also I simply removed for now the "HLTLevel1Activity.cc" module.

With all of these modules crippled to stop consuming l1extra and output empty collections, the HLT step now runs in standard sequences. I looked into these producers except HLTLevel1Activity and we should be able to just switch in the new formats.

So total tally I can see is one module completely rewritten (first pass almost done) and 5 modules need to modified once this is complete.

mulhearn commented 8 years ago

Oooof... I hope this is close to bottoming out: I still keep finding more changes needed. It seems we haven't hooked up BxVector to work with edm::Ref yet. This functionality is definitely needed to play nice with HLT.

mulhearn commented 8 years ago

BxVector is fixed to work with edm::Ref now...

mulhearn commented 8 years ago

@rekovic this should be a good start, I'll start a topic branch that includes my recent progress.

rekovic commented 8 years ago

@mulhearn Lets do it.

mulhearn commented 8 years ago

I got a bunch of core dumps while debugging this... I changed all the build files which I don't want to push but I think the correct recipe to just do this centrally is to edit the file:

$CMSSW_RELEASE_BASE//config/BuildFile.xml

to include:

<Flags CXXFLAGS="-g"/>

and this option will applied globally to all checked out packages.

mulhearn commented 8 years ago

I believe one of the problem spots is here:

  DataFormats/HLTReco/interface/TriggerFilterObjectWithRefs.h

which I started but didn't finish debugging:

-      collectionTags_.push_back(collectionTag.encode());
+      std::string tmp = collectionTag.encode();
+      //std::cout << collectionTags_ << "\n";
+      std::cout << "SIZE:  " << collectionTags_.size() << "\n";
+      collectionTags_.push_back(tmp);
mulhearn commented 8 years ago

Leaving out of branch some changes I made to these files, where were just to shut them up by commenting out everything...

#   modified:   HLTrigger/Muon/src/HLTMuonL1Filter.cc
#   modified:   RecoEgamma/EgammaHLTProducers/interface/HLTRecHitInAllL1RegionsProducer.h
#   modified:   RecoMuon/L2MuonSeedGenerator/src/L2MuonSeedGenerator.cc
#   modified:   RecoTauTag/HLTProducers/src/CaloTowerCreatorForTauHLT.cc
mulhearn commented 8 years ago

The initial comment on this githup issue has been updated to include the recipe for compiling and running the new topic branch, and summary of status.

rekovic commented 8 years ago

In DataFormats/HLTReco/interface/TriggerRefsCollections.h there is only one method that adds l1t:: objects.

void addObject(int id, const l1t::MuonRef& ref) {
  l1tmuonIds_.push_back(id);
  l1tmuonRefs_.push_back(ref);
}

Need to add methods for l1t::Egamma, l1t::Jet, l1t::Tau, l1t::EtSum Is this the whole list of l1t objects?

mulhearn commented 8 years ago

That'll do it!

rekovic commented 8 years ago

Made changes to extend functionality. Apart from l1t::Muon, we now have: EGamma, Jet, Tau, EtSum.

When putting above l1t candidates for HLT seeds, I used the existing HLT trigger types respectively: TriggerL1NoIsoEG,
TriggerL1CenJet, TriggerL1TauJet, TriggerL1ETT.

// Example For EGamma filterproduct.addObject(trigger::TriggerL1NoIsoEG, myref); // Temp just use NoIsoEG

Not sure if this OK. Need to check with with M.Gruenwald or A.Bocci.

If compiled with 

 USER_CXXFLAGS="-g -D=EDM_ML_DEBUG" scram b -j 8

running the test job

cmsRun L1Trigger/L1TCommon/test/devHLT.py

gives output log files critical.log and detailedInfo.log.
detailedInfo.log has output of L1T candidate seeds. PT is currently always zero. Did not look into why this is.

mulhearn commented 8 years ago

Andrea provides a useful set of questions to help organize this effort:

0. summarise the changes to the L1 software with respect to Legacy / Stage 1

My naïve guess is that all the interfaces to the L1 object have changed: new CondFormats, new DataFormats, etc.
I think that it may be helpful to have a brief summary or "translation table", to understand what conditions and objects were used for the Legacy trigger, and what are available now.

|   old C++ object   |   new C++ object   |   what is is / does   |

For example I would find it useful to understand what happens to 
  - the L1GtTriggerMenu conditions object, used to associate the trigger names to the bits; 
  - the L1GtTriggerMask conditions object, used for the trigger masks;
  - the L1GlobalTriggerReadoutRecord object, used to store the trigger results (before / after masks?)
  - the L1GlobalTriggerObjectMapRecord object, used to associate the triggers and the candidates
  - the l1extra::* objects, i.e. the L1 trigger candidates, and the additional quality flags they referenced (e.g. the L1MuGMTExtendedCand use by the l1extra::L1MuonParticle)

Some of these objects are clearly internal to the L1 software - the HLT does not need one to one replacements, just to maintain the overall functionalities - which leads to the next point.

1. identify all CMSSW modules used at HLT that need to be updated

In general terms, HLT uses
  - the L1 trigger decisions,
  - the L1 trigger candidates (the old "extra particles") and their qualities,
  - and the mapping between triggers and candidates (the old "object map").

Given the changes from 0. above, what HLT modules need to be updated, rewritten, or replaced with new modules ?

Basically every HLT path begins by unpacking and reading the L1 results and candidates:
  - L1 unpackers (L1GlobalTriggerRawToDigi, L1TRawToDigi, L1TCaloUpgradeToGCTConverter)
  - L1 producers of the mapping and candidates (L1GlobalTrigger, L1ExtraParticlesProd)
  - L1 "seed" filter, that checks if the given combination of L1 trigger fired, and produces the associated list of L1 candidates (HLTLevel1GTSeed)

How is this seqeunce going to change ?

Then, there are some specific use cases to be identified and updated. A few examples that come to mind are
  - the HLTMuonL1Filter is capable of applying further quality cuts to the L1 muon candidates
  - the L2MuonSeedGenerator uses the L1 muon candidates to seed the L2 reconstruction
  - the HLTEcalRecHitInAllL1RegionsProducer and EgammaHLTCaloTowerProducer use the L1 e/g candidates to select the regions to reconstruct
  - the HLTEgammaL1MatchFilterRegional matches the L1 and HLT e/g candidates (to avoid "volunteers")

Other modules that use the L1 candidates (I did not have a thorough look at them) are
  - CaloTowerCreatorForTauHLT
  - EgammaHLTCaloTowerProducer
  - HLTL1MuonSelector
  - HLTPFJetL1MatchProducer
  - IsolatedPixelTrackCandidateProducer

Note that I'm not saying that you need to migrate all these modules - but you need to provide enough information that the HLT developers can make the changes.

2. identify all other CMSSW modules NOT used at HLT that need to be updated, rewritten, or replaced with new modules

Well, this will need to happen at some point - but I won't include in the "urgent to do list". 
I just mean that once all HLT-related modules have been migrated, the work is not over.

3. identify the dependencies and priorities for the changes

Basically, what needs to be done first, what has to come later, etc.

Probably the earliest deadline is the MGWR in the second half of February: by then we need to be able to unpack the L1 results and filter events based on them.

4. Legacy vs. Stage 2 software

One last thing to consider is how the HLT modules should deal with the Legacy/Stage 1 vs. the Stage 2 software, as in principle, we may need to be able to run both in parallel.
Possible ideas:
  - we could have two copies of th HLT modules (for example via template specialisations) to deal with Legacy vs  Stage 2
  - we could foresee a way to translate the Legacy objects into the Stage 2 counterparts
  - ...
mulhearn commented 8 years ago

Please see also cms-sw#12994 for commit 78550a4d724115b5120b606cabc860fff7c6c42d and e8d283c53cf576e02638efd5fa1c98bfd1f1ecd3 which provide the new HLT enums.

mulhearn commented 8 years ago

Answer to Andrea point 4 is we definitely will provide a module that replaces L1ExtraProducer with something that outputs Legacy/Stage-1 output in the upgrade formats.

mulhearn commented 8 years ago

Q: summarise the changes to the L1 software with respect to Legacy / Stage 1

Emulator Changes

Almost the entire L1T emulation sequence has been replaced for Stage-2. I'll restrict most of my answer to the changes effecting consumers of L1 output, but the full details of which modules are running in the Stage-2 vs Stage-1 vs legacy are contained in the new era-driven configuration of the L1T-Emulation, which now cleanly switches between the 3-stages. The top-level config here:

https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/L1Trigger/Configuration/python/SimL1Emulator_cff.py

Simply calls the subsytem scripts here (that contain all the details):

https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/L1Trigger/L1TGlobal/python/simDigis_cff.py https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/L1Trigger/L1TCalorimeter/python/simDigis_cff.py https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/L1Trigger/L1TMuon/python/simDigis_cff.py

The most important module for HLT will be this one:

https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/L1Trigger/L1TGlobal/python/simGtStage2Digis_cfi.py

The upgrade GT emulator, which you will need to re-emulate in order to obtain the critical L1 object map, which maps particular L1 objects to the particular bit that they could contribute to firing.

This new module of course retrieves new inputs, so there are corresponding updates to the unpacker which are needed. These are in the new era-driven L1TRawToDigi sequence here:

https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/L1Trigger/Configuration/python/L1TRawToDigi_cff.py

For re-emulating L1TGlobal you will need to unpack "caloStage2Digis + gmtStage2Digis".

L1T OBJECT CLASSES

What are the inputs to L1TGlobal?

They are BX vectors (std::vectors that provide per bx iterators, bx=0 corresponding to the trigger bx) of upgrade format L1T objects: EGamma, Tau, Jet, EtSum, Muon:

  BXVector<l1t::EGamma> 
  BXVector<l1t::Tau>  
  BXVector<l1t::Jet> 
  BXVector<l1t::EtSum> 
  BXVector<l1t::Muon>

Unlike the legacy system, where a separate step was needed to convert emulator objects (with hardware values) to physical values (e.g. pt in GeV), the upgrade objects all contain both the digital hardware values and the values at physical scale. So you can now access the physical four vectors directly, via the L1Candidate interface which all of these objects satisfy:

https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/DataFormats/L1Trigger/interface/L1Candidate.h

Note in particular, that the L1Candidate is a reco::LeafCandidate, so in most cases the L1Candidates will be a drop in replacement for the legacy (deprecated) L1Extra Particle class.

If you need object specific quanities (e.g. Muon quality) they are via the object dependent derived classes:

https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/DataFormats/L1Trigger/interface/Jet.h https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/DataFormats/L1Trigger/interface/Muon.h https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/DataFormats/L1Trigger/interface/Sum.h https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/DataFormats/L1Trigger/interface/Tau.h https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/DataFormats/L1Trigger/interface/EGamma.h

The BX vector class is here:

https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/DataFormats/L1Trigger/interface/BXVector.h

but note that L1 Seeds are provided to you in the same manner as for the legacy system, as std::vectors of edm::Ref to the objects (e.g. std::vector< edm::Refl1t::Jet >.

Again, since this is essentially std::vector<edm::Ref< DERIVED FROM reco::LeafCandidate> > exactly as for legacy system, in most usage cases, the seeds should be a drop in replacement.

GLOBAL TRIGGER OUTPUT

What does this new L1TGlobal emulator produce?

GlobalAlgBlkBxCollection NEW GlobalExtBlkBxCollection NEW L1GlobalTriggerObjectMapRecord SAME FORMAT AS FOR LEGACY

Which are all defined here:

https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/DataFormats/L1TGlobal/interface/GlobalExtBlk.h https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/DataFormats/L1TGlobal/interface/GlobalAlgBlk.h https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/DataFormats/L1GlobalTrigger/interface/L1GlobalTriggerObjectMap.h

The first two objects are the replacement for the legacy "L1GlobalTriggerReadoutRecord". It is now broken into two parts, one for L1 algorithms (Alg), and one for triggers based on external (Ext) conditions.

The L1GlobalTriggerObjectMapRecord is the same format as for legacy, and will continue to provide the (emulated) mapping of which input L1T objects would fire a particular bit.

CONDITION FORMATS

How are the conditions for the new L1TGlobal set?

At the version of the code you will be give, these are all set via local XML file, or CSV file for the case of masks and prescale.

There is meanwhile an intense effort to provide updated CondFormats objects which is nearly converging. We have a new external library which handles the mapping of XML files to objects, which have (bitwise) aliases in CondFormats. At some point "soon", the configuration of L1TGlobal will be via these new CondFormats objects. You can follow this issue here:

https://github.com/cms-l1t-offline/cmssw/issues/71

and see the eventual CondFormat objects here:

https://github.com/cms-l1t-offline/cmssw/commit/08b95b5acd59968791cc57200252d0086e2025d0

mulhearn commented 8 years ago

Q: identify all CMSSW modules used at HLT that need to be updated

Scale of Problem

Ooof. I'm not sure I can give a complete answer, since I only know what currently breaks. I'm not entirely sure what else will break once these are all fixed, or which modules are suffering in silence.

However, I did do a simple test where I quieted all modules in standard HLT sequence that complained, after I swapped the legacy L1 seed producer (HLTLevel1GTSeed) with our drop in replacement (HLTL1TSeed) described in more detail below. For what it is worth the complete set of modules that complained were:

HLTrigger/Muon/src/HLTMuonL1Filter.cc RecoEgamma/EgammaHLTProducers/interface/HLTRecHitInAllL1RegionsProducer.h RecoMuon/L2MuonSeedGenerator/src/L2MuonSeedGenerator.cc RecoTauTag/HLTProducers/src/CaloTowerCreatorForTauHLT.cc

which a quick scanned looked like its a matter of replacing l1Extra with the drop in upgrade formats. I didn't look at all yet at this guy, but just removed him the HLT standard sequence for my test:

HLTLevel1Activity.cc

Comparing with your list, I see that my simple test did not catch these additional modules, which I'll have to look trhough as well:

HLTEcalRecHitInAllL1RegionsProducer EgammaHLTCaloTowerProducer HLTEgammaL1MatchFilterRegional EgammaHLTCaloTowerProducer HLTL1MuonSelector HLTPFJetL1MatchProducer IsolatedPixelTrackCandidateProducer

HLTL1TSeed

This is the main guy. This is intented as a drop in replacement for HLTLevel1GTSeed. As of today, v1.0 is available.

The module takes the upgrade inputs and provides the list of L1T seeds as an std::vector of edm::Refs to upgrade L1Objects. As these upgrade L1Objects satisfy the same reco::LeafCandidate interace as old L1Extra particles, this should largely be a drop in replacement for the downstream code as well.

There is only one major caveat: the v1.0 of this module provides all of the L1T seeds. An update will re-implement the functionality which provides the subset of seeds firing a particular list of L1 bits.

L1TGlobalAnalyzer

This module:

https://github.com/cms-l1t-offline/cmssw/blob/l1t-integration-pass3-80x/L1Trigger/L1TGlobal/plugins/L1TGlobalAnalyzer.cc

provides the functionality of the:

https://twiki.cern.ch/twiki/bin/view/CMSPublic/SWGuideL1TriggerL1GtUtils

but is not yet a drop-in replacement.

Note that the new version will not rely on conditions at the time of use, but will instead use a mapping of trigger bit to strigger name which is stored in the Event record.

This functionality is not used directly by HLT so hasn't been prioritized over the HLTLevel1GTSeed module.

mulhearn commented 8 years ago

Q: identify all other CMSSW modules NOT used at HLT that need to be updated, rewritten, or replaced with new modules

I guess the main one here is the replacement for L1GtUtils which has widespread use in CMSSW. Not sure yet what else remains on this list. We'll see which workflows downstream of HLT break once we have that working.

Q: identify the dependencies and priorities for the changes

Punting on this until we talk. I believe we've provided v1.0 of the most important module (HLTL1TSeed) but I don't know how to prioritize the remaining steps.

mulhearn commented 8 years ago

Q: Legacy vs. Stage 2 software

One last thing to consider is how the HLT modules should deal with the Legacy/Stage 1 vs. the Stage 2 software, as in principle, we may need to be able to run both in parallel. Possible ideas:

ANSWER: I infinitely prefer option 2. We'll provide a replacement for L1ExtraProducer which converts legacy objects into upgrade format, so that HLT can use the same interface for legacy system after the code updates. There is some complication here in that the legacy objects are broken into different number of collections (e.g. isoTaus and non-iso Taus, compared to upgrade, which simply has Taus.) This has consequences for interpreting the L1TObjectMap...

rekovic commented 8 years ago

Need to check functionality of TriggerReportSummary, as suggested by Martin G.

Martin-Grunewald commented 8 years ago

Hi,

I meant the trigger summary producers (which also pack up the L1 collections, and "passing object" indices per L1GTseed filter): https://github.com/cms-sw/cmssw/blob/CMSSW_8_0_X/HLTrigger/HLTcore/interface/TriggerSummaryProducerAOD.h https://github.com/cms-sw/cmssw/blob/CMSSW_8_0_X/HLTrigger/HLTcore/plugins/TriggerSummaryProducerAOD.cc and https://github.com/cms-sw/cmssw/blob/CMSSW_8_0_X/HLTrigger/HLTcore/interface/TriggerSummaryProducerRAW.h https://github.com/cms-sw/cmssw/blob/CMSSW_8_0_X/HLTrigger/HLTcore/plugins/TriggerSummaryProducerRAW.cc

mulhearn commented 8 years ago

The dev branch for L1TGlobal development with the utm library is here (for reference): https://github.com/cms-l1t-offline/cmssw/tree/l1t-global-jan-dev-CMSSW_8_0_0_pre4

rekovic commented 8 years ago

Working note:

I have got the HLTL1TSeed module to access L1GlobalTriggerObjectMapRecord, and loop over all its Maps, but it turns out it is not enough.

In Legacy L1/HLT interface one does need to be able to get the L1TriggerMenu from the EventSetup. Once menu is obtained, the algorithm and aliases maps are obtained

   const AlgorithmMap& algorithmMap      = l1GtMenu->gtAlgorithmMap();
   const AlgorithmMap& algorithmAliasMap = l1GtMenu->gtAlgorithmAliasMap();

which are then used to retrieve the GT decision bitNumbers for all seeding conditions. This bitNumber is then stored as the tokenNumber member variables of each L1GtLogicParser::OperandToken in m_l1AlgoLogicParser. This is the first update of m_l1AlgoLogicParser.

In the method seedsL1TriggerObjectMaps(....) there is a second update of m_l1AlgoLogicParser. The OperandToken.tokenNumbers are then used to get the decision of the GT decisionWord and store it in OperandToken.tokenResult for each OperandToken in the vector. If TRUE then the seeding objects for that OperanToken.tokenName are retrieved from the L1GlobalTriggerObjectMapRecord.

So, bottom line is, at the moment I need to access the L1TriggerMenu and GT decisonWord (i.e. L1GlobalTriggerReadoutRecord) to be able to continue.

fwyzard commented 8 years ago

@rekovic , don't you have all the necessary information and code available in the branch https://github.com/cms-l1t-offline/cmssw/tree/l1t-global-jan-dev-CMSSW_8_0_0_pre4 ?

mulhearn commented 8 years ago

That relies on utm being installed... and there's some integration work needed, but in principle, yes.

mulhearn commented 8 years ago

The new conditions we'll eventually have in event setup (based on UTM library) are here:

https://github.com/cms-l1t-offline/cmssw/blob/l1t-global-jan-dev-CMSSW_8_0_0_pre4/CondFormats/L1TObjects/interface/L1TUtmTriggerMenu.h

You can see this parallels the structure of the old TriggerMenu, but the low-level fields are slightly different. So the GtParser will need a corresponding update too.

blwiner commented 8 years ago

Perhaps this is known, but as I mentioned in the L1 meeting on Tuesday. L1TGlobalUtils pulls the L1 Menu out of the EventSetup and provides methods for getting bit number by trigger name, or trigger name by bit number, trigger bit decisions, etc. As an example, these Utils are used in GtRecordDump.

-Brian

On Thu, Jan 28, 2016 at 6:36 AM, mulhearn notifications@github.com wrote:

The new conditions we'll eventually have in event setup (based on UTM library) are here:

https://github.com/cms-l1t-offline/cmssw/blob/l1t-global-jan-dev-CMSSW_8_0_0_pre4/CondFormats/L1TObjects/interface/L1TUtmTriggerMenu.h

You can see this parallels the structure of the old TriggerMenu, but the low-level fields are slightly different. So the GtParser will need a corresponding update too.

— Reply to this email directly or view it on GitHub https://github.com/cms-l1t-offline/cmssw/issues/103#issuecomment-176137444 .

Prof. Brian L. Winer 
              Department of Physics
           The Ohio State University
             191 W. Woodruff Ave
              Columbus, OH 43210

               (614) 292-8996 (Office)
               (614) 292-7370 (Lab)

mulhearn commented 8 years ago

Let's chat if possible...

mulhearn commented 8 years ago

@fwyzard @Martin-Grunewald v1.0 of the HLTL1TSeed module is finally available at: https://github.com/cms-l1t-offline/cmssw/tree/l1t-hlt-demo-v1-CMSSW_8_0_0_pre5 the demo code is:

cmsRun L1Trigger/L1TCommon/test/reEmulHLT.py
cmsRun L1Trigger/L1TCommon/test/runMcHLT.py

The first example re-Emulates the entire L1T from RAW and feeds it to the HLTSeedFilter. The second runs L1T on MC, starting just after GEN sequence. I can provide another version which packs the reEmulated L1T into RAW and unpacks. I can also provide other customizations at your request... this was just my best guess as to what would be useful. Unless we get requests we'll provide v2.0 in similar fashion.

@rekovic has been making good progress on the v2.0 which will not be far behind. It appears we'll be able to parse everything we need from the uGT Object map. This is simpler and independent of UTM (though even that is available in latest IBs)

mulhearn commented 8 years ago

@rekovic: this commit a031422b8e05b3f7ae13cacb132bda0c137eb805 extends the object map class to include an object type vector, and fills it during the emulation. I thought it rather silly to have only 95% of what we need contained an object designed for this single purpose. To incorporate into your working area, you would do:

git remote add cms-l1t-offline git@github.com:cms-l1t-offline/cmssw.git
git fetch cms-l1t-offline
git cms-addpkg DataFormats/L1GlobalTrigger
git cherry-pick a031422b8e05b3f7ae13cacb132bda0c137eb805
Martin-Grunewald commented 8 years ago

@mulhearn @rekovic

Thanks a lot! Overall we would need three use-case example, two HLT-specific:

1) running the minimal L1T needed for HLT, starting from RAW (ie, unpacker, object maps etc., to get both all L1T objects, and to feed HLTL1TSeed to get from it the specific L1T objects)

2) running the full L1T in re-emul mode, needed when we want to change the L1T. This may mean the GT only or also the calo/muon/... etc. Input would be (old) RAW (possibly 25ns stage1, or in future stage2), and packing the L1T results up in a new RAW without changing the other subdetectors (aka "L1REPACK")

3) standard sequences as to be run in MC production, such as GEN,SIM,DIGI,L1,DIGI2RAW,RAW2DIGI,L1RECO,RECO

Martin-Grunewald commented 8 years ago

Ah yes, and a PR, please!

mulhearn commented 8 years ago

It seems that it needs a rebase. Thanks for the details, I will work out the sequences too.

rekovic commented 8 years ago

@mulhearn With this fix we are still storing the old (Legacy) enum defined in DataFormats/L1GlobalTrigger/interface/L1GtObject.h We need new ones (Mu, EG, Jet, Tau...) for Stage2.

blwiner commented 8 years ago

This is being worked on.

Brian

Sent from my iPhone

On Jan 31, 2016, at 11:02 AM, rekovic notifications@github.com wrote:

@mulhearn https://github.com/mulhearn With this fix we are still storing the old (Legacy) enum defined in DataFormats/L1GlobalTrigger/interface/L1GtObject.h We need new ones (Mu, EG, Jet, Tau...) for Stage2.

— Reply to this email directly or view it on GitHub https://github.com/cms-l1t-offline/cmssw/issues/103#issuecomment-177535181 .

rekovic commented 8 years ago

@blwiner OK. For the time being working with hard wired conversion of types from Legacy to Stage2 in the interface module HLTL1TSeed.

rekovic commented 8 years ago

We have a working version of the interface module HLTL1TSeed which stores L1 objects for each seed of configurable L1SeedsLogicalExpression.

An example event from a test job:

HLTL1TSeed::hltFilter Dump TriggerFilterObjectWithRefs %MSG

HLTL1TSeed: seed logical expression = L1_SingleJet36 OR L1_SingleEG10

L1Mu seeds: 0 L1EG seeds: 1

    L1EG            pt = 24.5       eta =  -0.261   phi =  1.653

L1Jet seeds: 4

    L1Jet           pt = 102.5      eta =  -0.435   phi =  -1.15019
    L1Jet           pt = 79 eta =  -0.261   phi =  1.653
    L1Jet           pt = 54.5       eta =  -0.609   phi =  1.131
    L1Jet           pt = 41 eta =  -0.609   phi =  -2.45519

L1Tau seeds: 0 L1EtSum ETT seeds: 0 L1EtSum HTT seeds: 0 L1EtSum ETM seeds: 0 L1EtSum HTM seeds: 0

We are ready for another PR.

Martin-Grunewald commented 8 years ago

@rekovic Nice!

Martin-Grunewald commented 8 years ago

@mulhearn

Given the code ready for CMSSW, I'd now need some info on 3 aspects of python configurations: a) py config of modules run with each HLT path b) L1REAPCK configuration for cmsDriver c) L1T parts of standard cmsDriver steps DIGI, L1, DIGI2RAW, L1Reco, RECO

More details here:

ad a)

currently, the HLT menu (py config) runs the following in each HLT path. We'd need a replacement, in terms of a fully expanded and resolved py config (ie, no eras or so) containing module replacements:

process.HLTBeginSequence = cms.Sequence(
 process.hltTriggerType +
 process.HLTL1UnpackerSequence +
 process.HLTBeamSpot )

process.hltTriggerType = cms.EDFilter( "HLTTriggerTypeFilter",
    SelectedTriggerType = cms.int32( 1 )
)

process.HLTL1UnpackerSequence = cms.Sequence(
 process.hltGtDigis +
 process.hltCaloStage1Digis +
 process.hltCaloStage1LegacyFormatDigis +
 process.hltL1GtObjectMap +
 process.hltL1extraParticles )

process.hltGtDigis = cms.EDProducer( "L1GlobalTriggerRawToDigi",
    DaqGtFedId = cms.untracked.int32( 813 ),
    Verbosity = cms.untracked.int32( 0 ),
    UnpackBxInEvent = cms.int32( 5 ),
    ActiveBoardsMask = cms.uint32( 0xffff ),
    DaqGtInputTag = cms.InputTag( "rawDataCollector" )
)

process.hltCaloStage1Digis = cms.EDProducer( "L1TRawToDigi",
    lenSlinkTrailer = cms.untracked.int32( 8 ),
    lenAMC13Header = cms.untracked.int32( 8 ),
    CTP7 = cms.untracked.bool( False ),
    lenAMC13Trailer = cms.untracked.int32( 8 ),
    Setup = cms.string( "stage1::CaloSetup" ),
    InputLabel = cms.InputTag( "rawDataCollector" ),
    lenSlinkHeader = cms.untracked.int32( 8 ),
    FWId = cms.uint32( 4294967295 ),
    debug = cms.untracked.bool( False ),
    FedIds = cms.vint32( 1352 ),
    lenAMCHeader = cms.untracked.int32( 8 ),
    lenAMCTrailer = cms.untracked.int32( 0 ),
    FWOverride = cms.bool( False )
)

process.hltCaloStage1LegacyFormatDigis = cms.EDProducer( "L1TCaloUpgradeToGCTConverter",
    InputHFCountsCollection = cms.InputTag( 'hltCaloStage1Digis','HFBitCounts' ),
    InputHFSumsCollection = cms.InputTag( 'hltCaloStage1Digis','HFRingSums' ),
    bxMin = cms.int32( 0 ),
    bxMax = cms.int32( 0 ),
    InputCollection = cms.InputTag( "hltCaloStage1Digis" ),
    InputIsoTauCollection = cms.InputTag( 'hltCaloStage1Digis','isoTaus' ),
    InputRlxTauCollection = cms.InputTag( 'hltCaloStage1Digis','rlxTaus' )
)

process.hltL1GtObjectMap = cms.EDProducer( "L1GlobalTrigger",
    TechnicalTriggersUnprescaled = cms.bool( True ),
    ProduceL1GtObjectMapRecord = cms.bool( True ),
    AlgorithmTriggersUnmasked = cms.bool( False ),
    EmulateBxInEvent = cms.int32( 1 ),
    AlgorithmTriggersUnprescaled = cms.bool( True ),
    ProduceL1GtDaqRecord = cms.bool( False ),
    ReadTechnicalTriggerRecords = cms.bool( True ),
    RecordLength = cms.vint32( 3, 0 ),
    TechnicalTriggersUnmasked = cms.bool( False ),
    ProduceL1GtEvmRecord = cms.bool( False ),
    GmtInputTag = cms.InputTag( "hltGtDigis" ),
    TechnicalTriggersVetoUnmasked = cms.bool( True ),
    AlternativeNrBxBoardEvm = cms.uint32( 0 ),
    TechnicalTriggersInputTags = cms.VInputTag( 'simBscDigis' ),
    CastorInputTag = cms.InputTag( "castorL1Digis" ),
    GctInputTag = cms.InputTag( "hltCaloStage1LegacyFormatDigis" ),
    AlternativeNrBxBoardDaq = cms.uint32( 0 ),
    WritePsbL1GtDaqRecord = cms.bool( False ),
    BstLengthBytes = cms.int32( -1 )
)
process.hltL1extraParticles = cms.EDProducer( "L1ExtraParticlesProd",
    tauJetSource = cms.InputTag( 'hltCaloStage1LegacyFormatDigis','tauJets' ),
    etHadSource = cms.InputTag( "hltCaloStage1LegacyFormatDigis" ),
    isoTauJetSource = cms.InputTag( 'hltCaloStage1LegacyFormatDigis','isoTauJets' ),
    etTotalSource = cms.InputTag( "hltCaloStage1LegacyFormatDigis" ),
    centralBxOnly = cms.bool( True ),
    centralJetSource = cms.InputTag( 'hltCaloStage1LegacyFormatDigis','cenJets' ),
    etMissSource = cms.InputTag( "hltCaloStage1LegacyFormatDigis" ),
    hfRingEtSumsSource = cms.InputTag( "hltCaloStage1LegacyFormatDigis" ),
    produceMuonParticles = cms.bool( True ),
    forwardJetSource = cms.InputTag( 'hltCaloStage1LegacyFormatDigis','forJets' ),
    ignoreHtMiss = cms.bool( False ),
    htMissSource = cms.InputTag( "hltCaloStage1LegacyFormatDigis" ),
    produceCaloParticles = cms.bool( True ),
    muonSource = cms.InputTag( "hltGtDigis" ),
    isolatedEmSource = cms.InputTag( 'hltCaloStage1LegacyFormatDigis','isoEm' ),
    nonIsolatedEmSource = cms.InputTag( 'hltCaloStage1LegacyFormatDigis','nonIsoEm' ),
    hfRingBitCountsSource = cms.InputTag( "hltCaloStage1LegacyFormatDigis" )
)

process.HLTBeamSpot = cms.Sequence(
 process.hltScalersRawToDigi +
 process.hltOnlineBeamSpot )

process.hltScalersRawToDigi = cms.EDProducer( "ScalersRawToDigi",
    scalersInputTag = cms.InputTag( "rawDataCollector" )
)
process.hltOnlineBeamSpot = cms.EDProducer( "BeamSpotOnlineProducer",
    maxZ = cms.double( 40.0 ),
    src = cms.InputTag( "hltScalersRawToDigi" ),
    gtEvmLabel = cms.InputTag( "" ),
    changeToCMSCoordinates = cms.bool( False ),
    setSigmaZ = cms.double( 0.0 ),
    maxRadius = cms.double( 2.0 )
)

ad b)

cmsDriver-based L1REPACK step - defined in Configuration/StandardSequences/python/, files, and used by specifying, for example: --step L1REPACK:GT2

https://github.com/cms-sw/cmssw/blob/CMSSW_8_0_X/Configuration/StandardSequences/python/SimL1EmulatorRepack_GT_cff.py https://github.com/cms-sw/cmssw/blob/CMSSW_8_0_X/Configuration/StandardSequences/python/SimL1EmulatorRepack_GT1_cff.py https://github.com/cms-sw/cmssw/blob/CMSSW_8_0_X/Configuration/StandardSequences/python/SimL1EmulatorRepack_GT2_cff.py https://github.com/cms-sw/cmssw/blob/CMSSW_8_0_X/Configuration/StandardSequences/python/SimL1EmulatorRepack_GCTGT_cff.py

These are used in current RelVal and TSG tests.

Currently they run on "old" RAW, redo some part of L1T and pack things up in a new RAW; the cases considered and used are: (i) Run-1 data input, new Run-1 L1T menu, (ii) Run-1 or Run-2 data input, new Run-2 stage1 L1T menu (with or without rerunning stage 1 GCT). Which of these are being kept alive? This has consequences for RelVal tests.

For TSG testing and trigger development for 2016, we would need a new L1REPACK sequence, starting from 2015 Run-2 stage1 RAW data, rerun L1T GT with a new stage2 L1T menu (incl. rerun L1T stage2 calo /muon / whatever if needed), and make a new L1T stage2-like RAW.

These could use eras but we'd need to be able to extract a fully expanded and resolved py sequence definition after cmsDriver has made the workflow.

ad c)

other standard cmsDriver-based steps - the (L1T part of these) configs for these cmsDriver steps are probably handled via eras, here that is fine of course, but please let me know which era and where to find them.... DIGI, L1, DIGI2RAW, L1Reco, RECO are these alreay defined and integrated in CMSSW 80X for stage2 - under shich era name?

mulhearn commented 8 years ago

@Martin-Grunewald short answers for now: (a) understood: we will provide this today. It should not be too difficult since this is simply a copy of our already working Stage-2 sequence. (b) This isn't quite finished yet. I have a customization for re-Emulation working that includes Stage-2, as well as RawToDigi and DigiToRaw. But I haven't combined these yet into L1REPACK sequence. (c) I have not merged with official CMSSW any changes to our top-level standard sequence configs, but we have these locally, and I'm nearly ready to submit a PR which does so.

mulhearn commented 8 years ago

Not stalled. There's some problem already with (a) ... the new standard sequences I'm relying on aren't working since the move to pre6. I'm working on it.

Martin-Grunewald commented 8 years ago

@rekovic What is the status of the trigger summary producers? Did you start to work on it, or should I take a look (of course needing your help on the object interpretation, esp. global sums and MET-like etc.)?

rekovic commented 8 years ago

@Martin-Grunewald Hi Martin, Yes. I have adapted the TriggerSummaryProducerRAW and TriggerSummaryProducerAOD. The code is pushed 7a8b151ee3b84180ed975b340af09f9be128c5ec to my branch cms-l1t-offline:rekovic-hlt-80x.

The HLTL1TSeed module stores the TriggerTypes and Refs to Stage2 L1T passing objects together with the types into the TriggerFilterObjectWithRefs. In the method dumpTriggerFilterObjectWithRefs (TFOWR), the filter object is then used to access the stored objects and verify their values. The size of vectors holding Vids and Refs is correct for each object type.

When the TriggerFilterObjectWithRefs is accessed in the TriggerSummaryProducerRAW(AOD) it shows for each type the correct number of objects stored. That is to say the size of vector holding Vids is correct, but the vector holding Refs is empty.

This is contrary to what is verified in the module HLTL1T. I am trying to understand why is it that the Event does not properly store the Refs to BxVectors in the vector of TFOWR.

mulhearn commented 8 years ago

@Martin-Grunewald I'm about to post the answer for part (a). Its been tested that this new hlt unpacker snippet recaptures the simulated stage-2 digis exactly, but I haven't yet tested object map or uGT output extensively but these modules do also run in the sequence and produce the expected output. I'm going to move on to the L1REPACK (b) task and leave further testing for later.

mulhearn commented 8 years ago

The new sequence is defined and tested here: https://github.com/cms-l1t-offline/cmssw/commit/d71a370c5dd95c01ce21b9822ee0798a04bc99d1#diff-6ee14fd888f62527576427ebc129ffdcR44 For now you'll need to merge in cms-sw#13233 to include some bug-fixes since pre6, in order for this sequence to run.

mulhearn commented 8 years ago

(Now onto L1REPACK)

Martin-Grunewald commented 8 years ago

@mulhearn This part is problematic for us:

process.hltGtStage2ObjectMap = cms.EDProducer("l1t::GtProducer",
+    #TechnicalTriggersUnprescaled = cms.bool(False),
+    ProduceL1GtObjectMapRecord = cms.bool(True),
+    AlgorithmTriggersUnmasked = cms.bool(False),
+    EmulateBxInEvent = cms.int32(1),
+    L1DataBxInEvent = cms.int32(5),
+    AlgorithmTriggersUnprescaled = cms.bool(False),
+    ProduceL1GtDaqRecord = cms.bool(True),
+    GmtInputTag = cms.InputTag("hltGmtStage2Digis"),
+    extInputTag = cms.InputTag("gtInput"),
+    caloInputTag = cms.InputTag("hltCaloStage2Digis"),
+    AlternativeNrBxBoardDaq = cms.uint32(0),
+    #WritePsbL1GtDaqRecord = cms.bool(True),
+    TriggerMenuLuminosity = cms.string('startup'),
+     PrescaleCSVFile = cms.string('prescale_L1TGlobal.csv'),
+    PrescaleSet = cms.uint32(1),
+    BstLengthBytes = cms.int32(-1),
+    Verbosity = cms.untracked.int32(0)
+)

As this part (a) it has to live in ConfDB, please change the type of the module from "l1t::GtProducer" to something which does NOT contain a namespace:: as this can't be handled by ConfDB. I.e., please do a typedef to, say, "L1TGtProducer" or so, and put that type into the FW macro to declare it as a plugin.

Further, I see that this module has the following parameters where I need more details:

mulhearn commented 8 years ago

We will fix the l1t namespace issue. extInputTag: we don't yet have an emulator for external conditions, but your sequence can certainly use the unpacked version, will fix that. We will make 'startup' disappear.