Closed sroychow closed 7 months ago
Files at
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/SiPixelDynamicInefficiency_L1_col100.db
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/SiPixelDynamicInefficiency_L1_col80.db
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/SiPixelDynamicInefficiency_L1_col85.db
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/SiPixelDynamicInefficiency_L1_col90.db
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/SiPixelDynamicInefficiency_L1_col95.db
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/SiPixelDynamicInefficiency_L2_col100.db
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/SiPixelDynamicInefficiency_L2_col80.db
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/SiPixelDynamicInefficiency_L2_col85.db
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/SiPixelDynamicInefficiency_L2_col90.db
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/dcol_scan_L1_L2/SiPixelDynamicInefficiency_L2_col95.db
For the record here are plots obtained with the following
that validate the internal content of the payloads posted at https://github.com/CMSTrackerDPG/Tasks/issues/1#issuecomment-1400551676
Sample production request:- https://its.cern.ch/jira/browse/PDMVMCPROD-78
The location of the PixelHistoMaker output files on the CERN EOS:
Data (runs 362433 to 362657 from /Muon/Run2022G-SiPixelCalSingleMuonTight-PromptReco-v1/ALCARECO
):
/eos/user/t/tbevilac/SWAN_projects/EPR/inputs/merged_runs362433To362657.root
MC:
/eos/cms/store/group/dpg_tracker_pixel/comm_pixel/PU_MCs/PU_MC_L1_100/
/eos/cms/store/group/dpg_tracker_pixel/comm_pixel/PU_MCs/PU_MC_L1_95/
/eos/cms/store/group/dpg_tracker_pixel/comm_pixel/PU_MCs/PU_MC_L1_90/
/eos/cms/store/group/dpg_tracker_pixel/comm_pixel/PU_MCs/PU_MC_L1_85/
/eos/cms/store/group/dpg_tracker_pixel/comm_pixel/PU_MCs/PU_MC_L1_80/
The above files are being copied to the CERN EOS.
First pass of results
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/validation/SiPixelDynamicInefficiency_PhaseI_2023_v1.db
I requested transfer of the produced RelVals to T2_HU_Budapest to protect them from being automatically deleted based on the standard 3-month lifetime for RelVals. Below is the list of Rucio rule IDs:
Rucio Rule ID Dataset
21411354388e4456a74e3098421f05bb cms:/RelValZMM_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_forPixelIneff_L2_col100_v1_MCProd78_v2-v1/GEN-SIM-RECO
286b02e2dcba4d8ab362268ef1e18614 cms:/RelValZMM_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_forPixelIneff_L2_col95_v1_MCProd78_v2-v1/GEN-SIM-RECO
569a7d2824214592822cab42f6d8c20a cms:/RelValZMM_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_forPixelIneff_L2_col85_v1_MCProd78_v2-v1/GEN-SIM-RECO
5a42c4b47c1d4ff0a325ba745730a761 cms:/RelValZMM_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_v1_MCProd78_v2-v1/GEN-SIM-RECO
61d899fe03ba47f785d64e352fd9d928 cms:/RelValZMM_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_forPixelIneff_L2_col80_v1_MCProd78_v2-v1/GEN-SIM-RECO
88bac8449646444693f460ee2a858865 cms:/RelValZMM_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_forPixelIneff_L2_col90_v1_MCProd78_v2-v1/GEN-SIM-RECO
debe319288a64dcb963d75c2532c0a01 cms:/RelValZMM_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_forPixelIneff_L1_col85_v1_MCProd78_v2-v1/GEN-SIM-RECO
e08e24659f4c451c8bd8f539bdce4362 cms:/RelValZMM_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_forPixelIneff_L1_col100_v1_MCProd78_v2-v1/GEN-SIM-RECO
ef4f978c2cb94b27a7d4d48980f646e6 cms:/RelValZMM_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_forPixelIneff_L1_col80_v1_MCProd78_v2-v1/GEN-SIM-RECO
f6d5f9b40c994494a008a826d5eb834b cms:/RelValZMM_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_forPixelIneff_L1_col90_v1_MCProd78_v2-v1/GEN-SIM-RECO
fd0ccdbf5b964888889b516af05e92c3 cms:/RelValZMM_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_forPixelIneff_L1_col95_v1_MCProd78_v2-v1/GEN-SIM-RECO
Requested GT creation with the preliminary measurement https://cms-talk.web.cern.ch/t/request-creation-of-a-gt-with-preliminary-pixel-dynamic-inefficiency-settings/20385
For the record PI plot of the PU factors for the new tag:
Relvals requested here:-
I had a very quick look to the RelValTTbar_SemiLeptonic
sample requested in https://its.cern.ch/jira/browse/PDMVRELVALS-188 and honestly I don't understand what's going on
I wanted to check, what to expect from the previous version of the .db and I've realized the following mistake... (In short: the dyn.ineff. factors are missing the x^4 fit parameter factor)
I've been using pol4 for the fitting the dcol eff vs inst. lumi. plot, and I've fixed the parameter 0 to 1 (to have 100% eff at 0 inst. lumi). And when I was printing out the efficiency factors, I was using the number GetNumberFreeParameters() to loop on all parameters. But since I've fixed 1 parameters, this was missing the last parameter... I should've realized my mistake at many occasions, sorry...
So this validation is useless, since the function with only 4 parameters looks like this
@bartokm
So this validation is useless, since the function with only 4 parameters looks like this
this explains why the new efficiency is better than the old one.
I posted earlier in the thread a plot from the PI about the PU factors (see https://github.com/CMSTrackerDPG/Tasks/issues/1#issuecomment-1432743841). From there you can see there are only 4 parameters. Why wasn't it noticed before?
I've checked your plot, but I've just didn't realize there is 1 factor missing. I'm sorry.
@bartokm @ferencek @mroguljic In the discussion from last week, it was suggested that we validate the payloads(both with and without module dependence). I propose the following action items for today
For tomorrow, we discuss the results for module dependency.
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/validation/SiPixelDynamicInefficiency_PhaseI_2023_v1_fix.db
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/validation/SiPixelDynamicInefficiency_PhaseI_2023_v1_fix.db
and here as an appetizer is the plot for this payload:
(working on the plot for the polynomial representation)
Hi @sroychow @bartokm,
do I understand correctly that this bug affects only the SiPixelDynamicInefficiency
that you wanted to include in the reDigi of the Run3Winter23
?
I mean the current 12_6_X GEN-SIM-RAW should be "ok"
/SinglePionGun_E0p2to200/Run3Winter23Digi-EpsilonPU_126X_mcRun3_2023_forPU65_v1-v1/GEN-SIM-RAW
(they should not have the SiPixelDynamicInefficiency
at all)
PS. 126X_mcRun3_2023_forPU65_v1
contains SiPixelDynamicInefficiency_PhaseI_v9 as SiPixelDynamicInefficiencyRcd
(no idea of what it means, I only know that the payload was inserted on 2021-07-07)
@silviodonato
(they should not have the SiPixelDynamicInefficiency at all)
they do have it, but it's the "naive" linear extrapolation from the Run-2 pre-kink data, which should be underestimating the real inefficiency at high PU.
@sroychow the new GT has been created.
(working on the plot for the polynomial representation)
still a bit rough at the edges (need to map the packed DetId to a human readable location in the detector), but here are the validation plots:
SiPixelDynamicInefficiency_phase1_2023_v1_fix
(new fixed payload):
SiPixelDynamicInefficiency_phase1_2023_v1
(bugged payload):
SiPixelDynamicInefficiency_PhaseI_v9
(for reference the old payload currently in GlobalTag):
New payload
New GT: 124X_mcRun3_2022_realistic_postEE_forPixelIneff_v3
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/validation/SiPixelDynamicInefficiency_PhaseI_2023_v2.db
I checked the polynomial representation:
barring some mistakes on my side, layer2 (i.e. region3 in the plot) looks fishy?
Indeed it looks bad... Let me double check
Wonderful I discovered another bug in my fitter code. Thanks Marco for your plots. In short: I've actually fitted layer 2 with pol2 and pol3, this was shown yesterday. (pol2 looks so close to linear, that I haven't realized the mistake). My suggestion: fix the .db with pol3 factors. Do you agree?
The only pol2 I see in Marco's plots is the one for region 3 but it does not look very linear. So I guess this is not the one that was shown yesterday which indeed looked very linear.
On Thu, Feb 23, 2023, 08:57 bartokm @.***> wrote:
Wonderful I discovered another bug in my fitter code. Thanks Marco for your plots. In short: I've actually fitted layer 2 with pol2 and pol3, this was shown yesterday. (pol2 looks so close to linear, that I haven't realized the mistake). My suggestion: fix the .db with pol3 factors. Do you agree?
— Reply to this email directly, view it on GitHub https://github.com/CMSTrackerDPG/Tasks/issues/1#issuecomment-1441331281, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFIWKKZ2ZIOUZFLVWSEVG3WY4J7FANCNFSM6AAAAAAUD7RKKM . You are receiving this because you were mentioned.Message ID: @.***>
What you see in Marco's plot is the pol3 fit without the x^3 parameter. What I've shown yesterday that is actually pol2 and pol3 (with all parameters).
Ok, thanks for the clarification. I agree with your proposal to fix v2 using pol3 with all the parameters.
On Thu, Feb 23, 2023, 09:18 bartokm @.***> wrote:
What you see in Marco's plot is the pol3 fit without the x^3 parameter. What I've shown yesterday that is actually pol2 and pol3 (with all parameters).
— Reply to this email directly, view it on GitHub https://github.com/CMSTrackerDPG/Tasks/issues/1#issuecomment-1441357808, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFIWKOOQUVXILZAA5QDXP3WY4MNFANCNFSM6AAAAAAUD7RKKM . You are receiving this because you were mentioned.Message ID: @.***>
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/validation/SiPixelDynamicInefficiency_PhaseI_2023_v2_fix.db
/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/validation/SiPixelDynamicInefficiency_PhaseI_2023_v2_fix.db
This one looks sensible:
getPayloadData.py --plugin pluginSiPixelDynamicInefficiency_PayloadInspector --plot plot_SiPixelDynamicInefficiencyPUParametrization --tag SiPixelDynamicInefficiency_PhaseI_2023_v2_fix --time_type Run --iovs '{"start_iov": "1", "end_iov": "1"}' --db sqlite_file:/afs/cern.ch/user/m/mbartok/public/pixel_dynamic_inefficiency_payloads/2023/validation/SiPixelDynamicInefficiency_PhaseI_2023_v2_fix.db --test ;
and inline with:
as shown yesterday, i.e. 85-ish % at the 2.5E34.
Here is the new GT: 124X_mcRun3_2022_realistic_postEE_forPixelIneff_v4
Hello @bartokm @ferencek @mroguljic Samples with 'v2' GT is available now, please start looking at the DQM plots
Samples with 'v2' GT is available now, please start looking at the DQM plots
I had a look to:
/RelValTTbar_SemiLeptonic_PU_13p6/CMSSW_12_4_11_patch3PU_124X_mcRun3_2022_realistic_postEE_forPixelIneff_v2_PDMVRELVALS188-v1/DQMIO
vs
/RelValTTbar_SemiLeptonic_PU_13p6/CMSSW_12_4_11_patch3-PU_124X_mcRun3_2022_realistic_postEE_v1_PDMVRELVALS188-v1/DQMIO
and they look very very similar: https://tinyurl.com/2fj225h6 Since the cmsDriver commands look OK to me (see message on JIRA) I am wondering if we are looking in a PU regime which is not very signficant:
should the samples be asked with a PU profile more similar to what we are expecting to see this year?
for record, here's the efficiency we see in the "naive" tracking analysis (N_valid / (N_missing + N_valid))
PU=60 in current simulation is 21.98 [10e33 cm^-2 s^-1], in new v2 payload is 20.58 [10e33 cm^-2 s^-1]. 21.98 -> 0.87621457 (inner) 0.91467012 (outer) (these are double column efficiency, not hit efficiency) 20.58 ->0.86862043 (inner) 0.91876270 (outer) So indeed they should be similar. At PU=70, the difference should be visible by eye: old -> 0.85559681 (inner) 0.90045754 (outer) new -> 0.76294018 (inner) 0.86917672 (outer)
FYI @ferencek @bartokm @mroguljic , new samples are available with Flat PU. Link to relval GUI
Please start checking. ALCARECO for the above are also available in case it is needed.
Samples with realistic 2023 PU profile are running as well.
We are trying to understand the change in the cluster sizes. If we look at the change of the size within the ladders of the inner shell (https://tinyurl.com/2ovxbxhf), we see that the size decreases for odd-numbered ladders and increases for even-numbered ladders. This would indicate that the dcol efficiency of payload 2023_v1_fix is higher for the outer ladders and lower for the inner ladders, compared to the dcol efficiency of the payload phase1_v9. Looking at the payloads, this may be the case.
If we look at the change of the size within the ladders of the outer shell (https://tinyurl.com/2jgss4el), we see the same - the size decreases for odd-numbered ladders and increases for even-numbered ladders. However, based on the online ladder coordinates, outer ladders of one half-shell should be odd-numbered and even-numbered for the other shell. This would indicate that the dcol efficiency for outer ladders increases in one half-shell and decreases in the other and vice-versa for the inner ladders. This really shouldn't be the case, so does anyone see a mistake in this reasoning?
We are trying to understand the change in the cluster sizes. If we look at the change of the size within the ladders of the inner shell (https://tinyurl.com/2ovxbxhf), we see that the size decreases for odd-numbered ladders and increases for even-numbered ladders. This would indicate that the dcol efficiency of payload 2023_v1_fix is higher for the outer ladders and lower for the inner ladders, compared to the dcol efficiency of the payload phase1_v9. Looking at the payloads, this may be the case.
I have two partially unrelated comments:
To add to Matej's post above, what prompted us to take a more detailed look at the plots Matej linked is that Marton informed us that he is confident that he mixed up inner and outer ladders in all his payloads produced so far so we wanted to be sure about this before requesting yet another sample production.
Looking at the 2023_v2_fix payload one would conclude all is fine regarding inner/outer ladders since the inner ladders have lower efficiency but looking at the older v9 payload the inner ladders have higher efficiency which is unphysical. @mmusich, how are you checking for the inner and outer ladders in the PayloadInspector?
Apart from the possible inner/outer mixup, the latest 2023_v2_fix payload goes in the right direction compared to the 2023_v1_fix and v9 payloads, as confirmed by the following plots Global eficiencies Eff vs PU for BPix L1 Eff vs PU for BPix L2
* I don't understand what's going on with the layer2 in the payload inspector plot for [2023_v1_fix](https://cms-conddb.cern.ch/cmsDbBrowser/payload_inspector/Prod/CMSSW_13_1_0_pre1/eyJwbG90cyI6MSwicGxvdDEiOnsidGFnIjoiU2lQaXhlbER5bmFtaWNJbmVmZmljaWVuY3lfcGhhc2UxXzIwMjNfdjFfZml4IiwicGxvdCI6InBsb3RfU2lQaXhlbER5bmFtaWNJbmVmZmljaWVuY3lQVVBhcmFtZXRyaXphdGlvbiIsInBsdWdpbiI6InBsdWdpblNpUGl4ZWxEeW5hbWljSW5lZmZpY2llbmN5X1BheWxvYWRJbnNwZWN0b3IiLCJ0eXBlIjoiSW1hZ2UiLCJpbnB1dF9wYXJhbXMiOnt9LCJpb3ZzIjp7InN0YXJ0X2lvdiI6IjEiLCJlbmRfaW92IjoiMSJ9fX0=) . The curve display doens't match the polynomial expansion written in the canvas and running locally I don't get that shape (baffled).
It looks like 2023_v1 suffers from the same issue I have no idea where the plotted curve comes from but the polynomial expansion written on the canvas is the one from the v9 payload
It looks like 2023_v1 suffers from the same issue
seems the webservice is glitching somehow, because locally you don't get these shapes :D
I have no idea where the plotted curve comes from but the polynomial expansion written on the canvas is the one from the v9 payload
what do you mean? v9
looks fine.
@ferencek
how are you checking for the inner and outer ladders in the PayloadInspector?
code is here:
Marton informed us that he is confident that he mixed up inner and outer ladders in all his payloads produced so far
curiously, I recently fixed what seemed to me a bug in this commit:
https://github.com/cms-sw/cmssw/commit/aa06c48a36008324b24408f137e04b2b3492c509
exactly because I was seeing larger inefficiency in the outer ladders. The fix still looks legit though.
I have no idea where the plotted curve comes from but the polynomial expansion written on the canvas is the one from the v9 payload
what do you mean?
v9
looks fine.
This was a somewhat useless comment since parameters for L2 in 2023_v1
and 2023_v1_fix
are the same as in v9
so this just means that the polynomial is correctly extracted and displayed but the plotted curve is somehow messed up.
@ferencek
how are you checking for the inner and outer ladders in the PayloadInspector?
code is here:
Is this code explicitly used when plotting the PU parametrization from the dynamic inefficiency payload? Or you get the info about what factors corresponds to inner/outer ladders in some other way? I am asking because if 2023_v2_fix
is plotted correctly, it would mean that in the old v9
payload factors were not assigned correctly.
Is this code explicitly used when plotting the PU parametrization from the dynamic inefficiency payload?
yes, but that's not all the code I use.
Or you get the info about what factors corresponds to inner/outer ladders in some other way?
well, it's a bit complicated because of how the information is stored in the payload. The class is here:
so breaking it up in pieces:
attachLocationLabel
. Code is here: https://github.com/cms-sw/cmssw/blob/master/CondCore/SiPixelPlugins/plugins/SiPixelDynamicInefficiency_PayloadInspector.cc#L798-L803. The attachLocationLabel
method makes use of the SiPixelPI::topolInfo
struct that I was pointing above https://github.com/CMSTrackerDPG/Tasks/issues/1#issuecomment-1456445597I am asking because if 2023_v2_fix is plotted correctly, it would mean that in the old v9 payload factors were not assigned correctly.
I can easily check if I am assigning the label correctly I you can give me a list of raw Ids for Layer1 internal / external ladders which come from an undisputed source (I was using some logic in the DQM code to determine if the ladders are inner or outer: https://github.com/cms-sw/cmssw/blob/cade7adf860867613704560511f827d78f76ab05/DQM/SiPixelPhase1Track/plugins/SiPixelPhase1TrackClusters.cc#L209-L212) but who knows if that's right :D
seems the webservice is glitching somehow, because locally you don't get these shapes :D
for the record, the webservice uses under the hood a singularity to el8 (while the default SCRAM_ARCH
one gets in lxplus is slc7_amd64_gcc10
):
[2023-03-06 16:39:47,637] [/data/services/cmsDbBrowser/app/payload_inspector/image_plot.py:40] - INFO: [PI] Getting image for plot type: Image
[2023-03-06 16:39:48,242] [/data/services/cmsDbBrowser/app/payload_inspector/plotting.py:130] - INFO: [PI] command to run: SCRAM_ARCH=el8_amd64_gcc11; export SCRAM_ARCH
cd /cvmfs/cms.cern.ch/${SCRAM_ARCH}/cms/cmssw/CMSSW_13_1_0_pre1/src
cmssw-el8 --command-to-run /bin/bash -c $@"source /cvmfs/cms.cern.ch/cmsset_default.sh;cmsenv;mkdir -p /tmp/pi_img_plots_2;cd /tmp/pi_img_plots_2;/afs/cern.ch/user/c/condbpro/public/BROWSER_PI/loadData.py --plugin pluginSiPixelDynamicInefficiency_PayloadInspector --plot plot_SiPixelDynamicInefficiencyPUParametrization --tag SiPixelDynamicInefficiency_phase1_2023_v1_fix --input_params '{}' --time_type Run --iovs '{\"start_iov\": \"1\", \"end_iov\": \"1\"}' --db Prod --suppress-output --image_plot"
that apparently makes the difference, because I am able to reproduce locally the bugged behaviour. Funky that it depends on the architecture :/
To study pixel efficiency at high pu, Zmumu samples are needed. The idea is the following:-
SiPixelDynamicInefficiency
124X_mcRun3_2022_realistic_postEE_Queue
. The idea is to re-use GEN-SIM from 124X production and re-run the DIGI with PU mixing & RECO steps. @ Sanjana