set up automated test workflows

[fwyzard]

Possible input samples for the automated tests:

• MC single mu, no pileup /RelValZMM_13/CMSSW_10_1_0_pre1-100X_upgrade2018_realistic_v10-v1/GEN-SIM-DIGI-RAW
• MC single mu, with pileup /RelValZMM_13/CMSSW_10_1_0_pre1-PU25ns_100X_upgrade2018_realistic_v10-v1/GEN-SIM-DIGI-RAW ?
• MC ttbar, with pileup /RelValTTbar_13/CMSSW_10_1_0_pre1-PU25ns_100X_upgrade2018_realistic_v10-v1/GEN-SIM-DIGI-RAW
• Run2017 data, SingleMuon dataset (e.g. run 306121, lumi 64-93)
• Run2017 data, JetHT dataset (e.g. run 306121, lumi 64-93)

Reference:

• 10824.5 pixel tracking only, from the reference release

Possible workflows:

• 10824.5 pixel tracking only
• 10824.7 pixel tracking only, with Riemann fit
• 10824.8 pixel tracking only, running on the GPU
• ? pixel tracking only, running on the GPU, with Riemann fit

Comparisons

• makeTrackValidationPlots.py (MC only)
• add pixel-only DQM sequences

[cmsbot]

A new Issue was created by @fwyzard Andrea Bocci.

can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[fwyzard]

A working version of the validation script now lives at https://github.com/cms-patatrack/patatrack-validation/blob/master/validate .

[fwyzard]

Still to do:

• [ ] add support for real data workflows
• [x] produce and upload nvprof profiles
• [x] rearrange the report by dataset
• [x] integrate with GitHub

[fwyzard]

Support for nvprof profiles is included via https://github.com/cms-patatrack/patatrack-validation/commit/637e93025ccf58a7977af55a2beb52ed88e95f36 .

Here is an example of the report including the profiles.

makeTrackValidationPlots.py plots

/RelValTTbar_13/CMSSW_10_1_0-PU25ns_101X_upgrade2018_realistic_v6-v1/GEN-SIM-DIGI-RAW

• tracking validation plots for workflow 10824.5
• tracking validation plots for workflow 10824.8
• tracking validation plots for workflow 10824.7

/RelValZMM_13/CMSSW_10_1_0-101X_upgrade2018_realistic_v6_resub-v1/GEN-SIM-DIGI-RAW

• tracking validation plots for workflow 10824.5
• tracking validation plots for workflow 10824.8
• tracking validation plots for workflow 10824.7

DQM GUI plots

/RelValTTbar_13/CMSSW_10_1_0-PU25ns_101X_upgrade2018_realistic_v6-v1/GEN-SIM-DIGI-RAW

• reference DQM plots for reference release, workflow 10824.5
• DQM plots for development release, workflow 10824.5
• DQM plots for development release, workflow 10824.8
• DQM plots for development release, workflow 10824.7
• DQM comparison for reference workflow 10824.5
• DQM comparison for workflow 10824.8
• DQM comparison for workflow 10824.7

/RelValZMM_13/CMSSW_10_1_0-101X_upgrade2018_realistic_v6_resub-v1/GEN-SIM-DIGI-RAW

• reference DQM plots for reference release, workflow 10824.5
• DQM plots for development release, workflow 10824.5
• DQM plots for development release, workflow 10824.8
• DQM plots for development release, workflow 10824.7
• DQM comparison for reference workflow 10824.5
• DQM comparison for workflow 10824.8
• DQM comparison for workflow 10824.7

nvprof/nvvp profiles

/RelValTTbar_13/CMSSW_10_1_0-PU25ns_101X_upgrade2018_realistic_v6-v1/GEN-SIM-DIGI-RAW

• reference profile for workflow 10824.5
• development profile for workflow 10824.5
• development profile for workflow 10824.8
• development profile for workflow 10824.7

/RelValZMM_13/CMSSW_10_1_0-101X_upgrade2018_realistic_v6_resub-v1/GEN-SIM-DIGI-RAW

• reference profile for workflow 10824.5
• development profile for workflow 10824.5
• development profile for workflow 10824.8
• development profile for workflow 10824.7

Logs

The full log is available at https://fwyzard.web.cern.ch/fwyzard/patatrack/pulls/1db4fa5f39344607676b85826ff612a865a3adff/log .