Maximum comparison: report step1 das query to align steps for all workflows, report summary as html table

gartung commented 2 months ago

please test

cmsbuild commented 2 months ago

A new Pull Request was created by @gartung for branch master.

@aandvalenzuela, @cmsbuild, @iarspider, @smuzaffar can you please review it and eventually sign? Thanks. @antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this. cms-bot commands are listed here

cmsbuild commented 2 months ago

cms-bot internal usage

smuzaffar commented 2 months ago

please test

smuzaffar commented 2 months ago

@gartung , it would be nice to have a single index.html which showing (in form of table) the max mem comparison of all relvals and possible setting a different color for failed one

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

Working on it.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

smuzaffar commented 2 months ago

@gartung , comparison failed

03:37:47 Traceback (most recent call last):
03:37:47   File "/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/cms-bot/comparisons/compare-maxmem-summary.py", line 226, in <module>
03:37:47     compare_maxmem_summary(
03:37:47   File "/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/cms-bot/comparisons/compare-maxmem-summary.py", line 43, in compare_maxmem_summary
03:37:47     max_mem_pr = max_memory_pr_dict[step].get("max memory used")
03:37:47 KeyError: 'step5'

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

cmsbuild commented 2 months ago

Pull request #2335 was updated.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

cmsbuild commented 2 months ago

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ffff10/41637/summary.html COMMIT: 6c99b1b484a0a985eff92a5c4679ad1bf17be71f CMSSW: CMSSW_14_2_X_2024-09-19-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/2335/41637/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 3 lines to the logs
Reco comparison results: 92 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3331158
DQMHistoTests: Total failures: 3205
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3327933
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 193 log files, 163 edm output root files, 44 DQM output files
TriggerResults: no differences found

cmsbuild commented 2 months ago

Pull request #2335 was updated.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

gartung commented 2 months ago

@smuzaffar How can I run black formatting check locally?

smuzaffar commented 2 months ago

@smuzaffar How can I run black formatting check locally?

you can locall install black e.g.

scram p CMSSW_14_2_0_pre1
cd CMSSW_14_2_0_pre1/
cmsenv
scram-venv
cmsenv
pip3 install black
which black

and then run black pointing to your cms-bot dir

black /cms-bot/directory

cmsbuild commented 2 months ago

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ffff10/41653/summary.html COMMIT: 4f24df815ec5b28130ea484f13c0cccfe50faa34 CMSSW: CMSSW_14_2_X_2024-09-19-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/2335/41653/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 5 lines to the logs
Reco comparison results: 100 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3331158
DQMHistoTests: Total failures: 3211
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3327927
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 193 log files, 163 edm output root files, 44 DQM output files
TriggerResults: no differences found

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

cmsbuild commented 2 months ago

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ffff10/41678/summary.html COMMIT: 108671337ab4a4721b509c21ec92b0ae850fdf49 CMSSW: CMSSW_14_2_X_2024-09-20-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/2335/41678/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 90 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3331158
DQMHistoTests: Total failures: 2874
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3328264
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 193 log files, 163 edm output root files, 44 DQM output files
TriggerResults: no differences found

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

cmsbuild commented 2 months ago

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ffff10/41682/summary.html COMMIT: 2d12683f42fe12b768cc77f6849ec3b51c528962 CMSSW: CMSSW_14_2_X_2024-09-20-1100/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/2335/41682/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 2 lines from the logs
Reco comparison results: 80 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3331158
DQMHistoTests: Total failures: 4598
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3326540
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 193 log files, 163 edm output root files, 44 DQM output files
TriggerResults: no differences found

cmsbuild commented 2 months ago

Pull request #2335 was updated.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

cmsbuild commented 2 months ago

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ffff10/41687/summary.html COMMIT: b534a252bed1f813cdf4ab557b8ffd60bbc278b9 CMSSW: CMSSW_14_2_X_2024-09-20-2300/el8_amd64_gcc12 User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cms-bot/2335/41687/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 1 lines from the logs
Reco comparison results: 21 differences found in the comparisons
DQMHistoTests: Total files compared: 44
DQMHistoTests: Total histograms compared: 3331158
DQMHistoTests: Total failures: 1388
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3329750
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 43 files compared)
Checked 193 log files, 163 edm output root files, 44 DQM output files
TriggerResults: no differences found

cmsbuild commented 2 months ago

Pull request #2335 was updated.

cmsbuild commented 2 months ago

Pull request #2335 was updated.

gartung commented 2 months ago

please test

cms-sw / cms-bot