search

umd-lhcb / lhcb-ntuples-gen

ntuples generation with DaVinci and in-house offline components
BSD 2-Clause "Simplified" License
1 stars 0 forks source link

Apply PID weights with pidcalib2 (w/o UBDT weights) for RDX #83

Closed yipengsun closed 2 years ago

yipengsun commented 3 years ago

We should use the recently released pidcalib2. A quick intro slides can be found here

PID cuts to apply: RDX run 2 with run 1 cuts

I checked both the run 2 reco script and Manuel's cuts document, and that seems to be the case

All binning variables:

Extracted from: https://gitlab.cern.ch/lhcb-rta/pidcalib2/-/blob/master/src/pidcalib2/binning.py

valid_particles = ["Pi", "K", "P", "Mu", "e"]

def p_binning(particle: str, low: float = 3000, high: float = 100000) -> List[float]:
    """Return a binning for the momentum.

    Args:
        particle: Particle type ["Pi", "K", ...]
        low: Optional. Lowest momentum [MeV]. Defaults to 3000.
        high: Optional. Highest momentum [MeV]. Defaults to 100000.
    """
    if particle not in valid_particles:
        log.error(f"'{particle}' is not a valid particle for P binning")
        raise KeyError

    bins = []
    if particle in {"Pi", "K", "P", "e"}:
        bins.append(low)
        bins.append(9300)  # R1 kaon threshold
        bins.append(15600)  # R2 kaon threshold
        # Uniform bin boundaries
        uniform_bins = np.linspace(19000, high, 16).tolist()  # type:ignore
        bins.extend(uniform_bins)
    elif particle == "Mu":
        bins = [
            low,
            6000,
            8000,
            10000,
            12000,
            14500,
            17500,
            21500,
            27000,
            32000,
            40000,
            60000,
            70000,
            high,
        ]
    return bins

def eta_binning(particle, low: float = 1.5, high: float = 5.0) -> List[float]:
    return list(np.linspace(low, high, 5))

def ntracks_binning(particle, low: float = 0, high: float = 500) -> List[float]:
    return [low, 50, 200, 300, high]

def nspdhits_binning(particle, low: float = 0, high: float = 1000) -> List[float]:
    return [low, 200, 400, 600, 800, high]

def trchi2_binning(particle, low: float = 0.0, high: float = 3.0) -> List[float]:
    return list(np.linspace(low, high, 4))

Binning variables for RDX run 2

We can't bin on :

So, we can only bin on:

yipengsun commented 3 years ago

pidcalib2 is actually quite fast. Running the demo script only took 15s:

suny@lxplus743 ~/eos/src/lhcb-ntuples-gen/run2-rdx/reweight/pid
❯ ./run2-rdx-demo.sh 
[I 210817 15:23:37 make_eff_hists:180] Running PIDCalib2 make_eff_hists with the following config:
[I 210817 15:23:37 utils:116] ===============
[I 210817 15:23:37 utils:119] sample         : Turbo16
[I 210817 15:23:37 utils:119] magnet         : down
[I 210817 15:23:37 utils:119] particle       : K
[I 210817 15:23:37 utils:119] pid_cuts       : ['DLLK>4.0']
[I 210817 15:23:37 utils:119] bin_vars       : ['P', 'ETA']
[I 210817 15:23:37 utils:119] output_dir     : pidcalib_output
[I 210817 15:23:37 utils:119] max_files      : 3
[I 210817 15:23:37 utils:119] verbose        : False
[I 210817 15:23:37 utils:119] version        : 1.0.0
[I 210817 15:23:37 utils:120] ===============
[W 210817 15:23:37 pid_data:362] You are using the --max-files option; it should be used only for testing
[I 210817 15:23:37 utils:453] Branches to be read: {'sWeight': 'probe_sWeight', 'DLLK': 'probe_PIDK', 'P': 'probe_P', 'ETA': 'probe_ETA'}
[I 210817 15:23:37 utils:454] 3 calibration files from EOS will be processed
[I 210817 15:23:47 utils:506] Processed 3/3 files                                                                                                                                                                                             
[I 210817 15:23:47 utils:587] 1089831/1173971 (92.8%) events passed binning range cut
[I 210817 15:23:47 utils:587] 885707/1089831 (81.3%) events passed 'DLLK>4.0' cut
[I 210817 15:23:47 make_eff_hists:216] Efficiency histograms saved to 'pidcalib_output/effhists-Turbo16-down-K-DLLK>4.0-P.ETA.pkl'

suny@lxplus743 ~/eos/src/lhcb-ntuples-gen/run2-rdx/reweight/pid 15s
yipengsun commented 3 years ago

Welp, pidcalib2 doesn't play well with our existing Python venv. I'll not install that locally for now. If we want to install it in the future, we need to create a separate venv for it (because it uses uproot3, we use uproot4, and we need to pack xrootd in nix).

Won't do these for now.

yipengsun commented 3 years ago

Welp, pidcalib2 doesn't play well with our existing Python venv. I'll not install that locally for now. If we want to install it in the future, we need to create a separate venv for it (because it uses uproot3, we use uproot4, and we need to pack xrootd in nix).

Actually, uproot3 and uproot can co-exist w/o problem, because they live in different name spaces. The only problem is the missing xrootd, which I can try pack it myself.

yipengsun commented 3 years ago

Also, I see a new package pidgen2 under lhcb-rta. From RD+'s trigger emulation note:

The emulation of the particle ID can be readily done with the PIDCalib package. The PIDcorr package, which corrects the existing simulation response cannot be used. However, the PIDgen package can be used without any problem. Correlations between PID variables are lost, which could be an issue for analyses training MVAs on simulation but is overall not a huge concern.

I'm wondering which one should we use? pidcalib2 or pidgen2 (which is very new and is perhaps unstable)?

yipengsun commented 3 years ago

In the end, I think pidcalib2 will be largely useless without EOS access because most of its scripts require access to existing PID ntuples.

One exception is pklhisto2root.py, which converts pickled boost histogram to ROOT's THND. But we can add this script to this repo so that we can do these conversions without installing pidcalib2.

yipengsun commented 3 years ago

So, for our isolation trackers, we apply NNN{K,pi,etc} cuts on them, which depends both on the PID variables and tracking variables.

How'd we deal with them in MC? Do we need to apply a PID weights for them as well? (Doubtful as if that's the case, we can't easily separate our samples into subsamples).

yipengsun commented 3 years ago

This is not a problem for our normal reweighting, because in the PID efficiency generation, we specified the true particle with --particle K, then in our MC truth matching, we require reconstructed K is a true K.

yipengsun commented 3 years ago

So, for our isolation trackers, we apply NNN{K,pi,etc} cuts on them, which depends both on the PID variables and tracking variables.

How'd we deal with them in MC? Do we need to apply a PID weights for them as well? (Doubtful as if that's the case, we can't easily separate our samples into subsamples).

We'll truth-match these tracks first, then apply PIDCalib-based weights (with a cut on NNk).

yipengsun commented 2 years ago

Some interesting comparison between pidcalib2, pidgen2, and the old PidGen: https://gitlab.cern.ch/lhcb-rta/pidgen2/-/tree/master/doc

yipengsun commented 2 years ago

With default binning scheme, there are 14 bins in P and 5 bins in ETA. 10 out of 52 bins are empty. These are:

bin idx: 0, 0 val: 0                                                                                                                                                                                                                           
bin idx: 0, 1 val: 0                                                                                                                                                                                    
bin idx: 0, 2 val: 0                                                                                                                                                                                    
bin idx: 0, 3 val: 0                                                                                                                                                                                    
bin idx: 0, 4 val: 0                                                                                                                                                                                    
bin idx: 1, 0 val: 0                                                                                                                                                                                    
bin idx: 1, 1 val: 0.764466                                                                                                                                                                                                         
bin idx: 1, 2 val: 0.766231                                                                                                                                                                                                         
bin idx: 1, 3 val: nan
bin idx: 1, 4 val: nan
bin idx: 2, 0 val: 0
bin idx: 2, 1 val: 0.770966
bin idx: 2, 2 val: 0.899823
bin idx: 2, 3 val: nan
bin idx: 2, 4 val: nan
bin idx: 3, 0 val: 0
bin idx: 3, 1 val: 0.80977
bin idx: 3, 2 val: 0.869584
bin idx: 3, 3 val: nan
bin idx: 3, 4 val: nan
bin idx: 4, 0 val: 0
bin idx: 4, 1 val: 0.829013
bin idx: 4, 2 val: 0.853972
bin idx: 4, 3 val: 0.79353
bin idx: 4, 4 val: nan
bin idx: 5, 0 val: 0
bin idx: 5, 1 val: 0.799171
bin idx: 5, 2 val: 0.842485
bin idx: 5, 3 val: 0.780866
bin idx: 5, 4 val: nan
bin idx: 6, 0 val: 0
bin idx: 6, 1 val: 0.815424
bin idx: 6, 2 val: 0.849175
bin idx: 6, 3 val: 0.75708
bin idx: 6, 4 val: nan
bin idx: 7, 0 val: 0
bin idx: 7, 1 val: 0.876123
bin idx: 7, 2 val: 0.877593
bin idx: 7, 3 val: 0.763988
bin idx: 7, 4 val: nan
bin idx: 8, 0 val: 0
bin idx: 8, 1 val: 0.923247
bin idx: 8, 2 val: 0.923672
bin idx: 8, 3 val: 0.795881
bin idx: 8, 4 val: 0.708212
bin idx: 9, 0 val: 0
bin idx: 9, 1 val: 0.949531
bin idx: 9, 2 val: 0.948424
bin idx: 9, 3 val: 0.824485
bin idx: 9, 4 val: 0.706749
bin idx: 10, 0 val: 0
bin idx: 10, 1 val: 0.956257
bin idx: 10, 2 val: 0.963354
bin idx: 10, 3 val: 0.858827
bin idx: 10, 4 val: 0.723521
bin idx: 11, 0 val: 0
bin idx: 11, 1 val: 0.961861
bin idx: 11, 2 val: 0.969651
bin idx: 11, 3 val: 0.897365
bin idx: 11, 4 val: 0.723541
bin idx: 12, 0 val: 0
bin idx: 12, 1 val: 0.95193
bin idx: 12, 2 val: 0.96763
bin idx: 12, 3 val: 0.926081
bin idx: 12, 4 val: 0.701767
bin idx: 13, 0 val: 0
bin idx: 13, 1 val: 0.922278
bin idx: 13, 2 val: 0.961626
bin idx: 13, 3 val: 0.936822
bin idx: 13, 4 val: 0.662721

This efficiency histogram is generated with pretty standard settings.

yipengsun commented 2 years ago

With a better formatting:

File: run2-rdx-2016-md-Mu-empty_bins.root, Histo: eff
  y \ x    0        1        2       3        4        5        6        7      8      9     10     11     12     13
-------  ---  -------  -------  ------  -------  -------  -------  -------  -----  -----  -----  -----  -----  -----
      0    0    0        0        0       0        0        0        0      0      0      0      0      0      0
      1    0    0.764    0.771    0.81    0.829    0.799    0.815    0.876  0.923  0.95   0.956  0.962  0.952  0.922
      2    0    0.766    0.9      0.87    0.854    0.842    0.849    0.878  0.924  0.948  0.963  0.97   0.968  0.962
      3    0  nan      nan      nan       0.794    0.781    0.757    0.764  0.796  0.824  0.859  0.897  0.926  0.937
      4    0  nan      nan      nan     nan      nan      nan      nan      0.708  0.707  0.724  0.724  0.702  0.663
yipengsun commented 2 years ago

I tried to apply that Mu PID weight to B0 -> D* Tau Nu sample:

> apply_histo_weight.py ../../../samples/Dst_D0--21_07_30--mc--Bd2DstTauNu--2016--md--py8-sim09j-dv45-subset.root run2-rdx-2016-md-Mu-empty_bins.root test.root 
Total event processed: 7343, bad: 987, nan: 791. Bad fraction: 13.4%
yipengsun commented 2 years ago

I feel the re-binning is going to be non-trivial, because with this small local sample, we already have a significant fraction of events that are in either 0-valued-bin or NaN-valued bin region.

Somehow the official PIDCalib samples doesn't cover a officially defined binning! I feel I should collect the required files and report this upstream.

yipengsun commented 2 years ago

Looking at all 3 PIDCalib histos generated with all PID sample ntuples and desired cuts, with binning in P-ETA:

K

η \ p [Mev] 0 (U) 1 (6150.0) 2 (12450.0) 3 (17300.0) 4 (21700.0) 5 (27100.0) 6 (32500.0) 7 (37900.0) 8 (43300.0) 9 (48700.0) 10 (54100.0) 11 (59500.0) 12 (64900.0) 13 (70300.0) 14 (75700.0) 15 (81100.:) 16 (86500.0) 17 (91900.0) 18 (97300.0) 19 (O)
0 (U) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 (1.9) 0 0.95 0.96 0.97 0.98 0.97 0.93 0.85 0.72 0.61 0.49 0.38 0.28 0.2 0.14 0.09 0.06 0.03 0.02 0
2 (2.8) 0 0.86 0.94 0.96 0.97 0.97 0.96 0.93 0.91 0.88 0.86 0.82 0.78 0.73 0.66 0.59 0.51 0.44 0.37 0
3 (3.7) 0 0.76 0.91 0.94 0.95 0.95 0.95 0.94 0.93 0.92 0.89 0.85 0.79 0.72 0.64 0.56 0.48 0.4 0.33 0
4 (4.6) 0 0.16 0.62 0.75 0.81 0.82 0.79 0.77 0.77 0.76 0.74 0.69 0.63 0.57 0.5 0.43 0.36 0.29 0.23 0
5 (O) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
> apply_histo_weight.py ../../../samples/Dst_D0--21_07_30--mc--Bd2DstTauNu--2016--md--py8-sim09j-dv45-subset.root run2-rdx-2016-md-K.root test.root -x "k_P" -y "ETA(k_P, k_PZ)"
Total event processed: 7343, bad: 372, nan: 0. Bad fraction: 5.1%

Pi

η \ p [Mev] 0 (U) 1 (6150.0) 2 (12450.0) 3 (17300.0) 4 (21700.0) 5 (27100.0) 6 (32500.0) 7 (37900.0) 8 (43300.0) 9 (48700.0) 10 (54100.0) 11 (59500.0) 12 (64900.0) 13 (70300.0) 14 (75700.0) 15 (81100.0) 16 (86500.0) 17 (91900.0) 18 (97300.0) 19 (O)
0 (U) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 (1.9) 0 0.89 0.94 0.98 0.99 0.98 0.96 0.93 0.89 0.85 0.82 0.81 0.82 0.82 0.83 0.84 0.87 0.89 0.91 0
2 (2.8) 0 0.89 0.95 0.97 0.98 0.97 0.96 0.94 0.93 0.93 0.92 0.92 0.91 0.91 0.9 0.89 0.88 0.88 0.88 0
3 (3.7) 0 0.87 0.92 0.96 0.97 0.97 0.96 0.96 0.95 0.95 0.94 0.93 0.91 0.9 0.89 0.88 0.87 0.87 0.86 0
4 (4.6) 0 0.92 0.88 0.89 0.91 0.91 0.91 0.91 0.91 0.91 0.91 0.9 0.88 0.87 0.86 0.85 0.85 0.85 0.85 0
5 (O) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
> apply_histo_weight.py ../../../samples/Dst_D0--21_07_30--mc--Bd2DstTauNu--2016--md--py8-sim09j-dv45-subset.root run2-rdx-2016-md-Pi.root test.root -x "pi_P" -y "ETA(pi_P, pi_PZ)"
Total event processed: 7343, bad: 302, nan: 0. Bad fraction: 4.1%

Mu

η \ p [Mev] 0 (U) 1 (4500.0) 2 (7000.0) 3 (9000.0) 4 (11000.0) 5 (13250.0) 6 (16000.0) 7 (19500.0) 8 (24250.0) 9 (29500.0) 10 (36000.0) 11 (50000.0) 12 (65000.0) 13 (85000.0) 14 (O)
0 (U) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
1 (1.9) 0 0.76 0.77 0.81 0.83 0.8 0.82 0.88 0.92 0.95 0.96 0.96 0.95 0.92 0
2 (2.8) 0 0.77 0.9 0.87 0.85 0.84 0.85 0.88 0.92 0.95 0.96 0.97 0.97 0.96 0
3 (3.7) 0 nan nan nan 0.79 0.78 0.76 0.76 0.8 0.82 0.86 0.9 0.93 0.94 0
4 (4.6) 0 nan nan nan nan nan nan nan 0.71 0.71 0.72 0.72 0.7 0.66 0
5 (O) 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
apply_histo_weight.py ../../../samples/Dst_D0--21_07_30--mc--Bd2DstTauNu--2016--md--py8-sim09j-dv45-subset.root run2-rdx-2016-md-Mu-empty_bins.root test.root -x "mu_P" -y "ETA(mu_P, mu_PZ)" 
Total event processed: 7343, bad: 987, nan: 791. Bad fraction: 13.4%

Note: nan is part of bad, so the over/underflow is 196/7343 ~ 2.7%.

yipengsun commented 2 years ago

I tried to used P-nTracks binning for the Mu:

[W 210923 18:31:09 utils:62] There are 13 empty bins in the total histogram! You might want to change the binning.
y \ x 1 (4500.0) 2 (7000.0) 3 (9000.0) 4 (11000.0) 5 (13250.0) 6 (16000.0) 7 (19500.0) 8 (24250.0) 9 (29500.0) 10 (36000.0) 11 (50000.0) 12 (65000.0) 13 (85000.0)
1 (25.0) 0.77 0.85 0.86 0.85 0.83 0.83 0.84 0.87 0.88 0.89 0.89 0.87 0.84
2 (125.0) 0.76 0.83 0.84 0.84 0.82 0.81 0.83 0.86 0.87 0.88 0.88 0.87 0.84
3 (250.0) 1.8 0.85 0.71 0.76 1.23 1.01 0.63 0.78 0.88 0.92 1.15 1.35 1.37
4 (400.0) nan nan nan nan nan nan nan nan nan nan nan nan nan

Trying to apply it:

> apply_histo_weight.py ../../../samples/Dst_D0--21_07_30--mc--Bd2DstTauNu--2016--md--py8-sim09j-dv45-subset.root effhist.root test.root -x "mu_P" -y "nTracks"                       
Total event processed: 7343, bad: 481, nan: 279. Bad fraction: 6.6%
yipengsun commented 2 years ago

Need to think about how to apply ProbNNk cuts on the isolation tracks.

yipengsun commented 2 years ago

Phoebe confirmed that she used 3D binning: P-ETA-nTracks.

yipengsun commented 2 years ago

Low P and high ETA seems like low-PT Mu. Maybe we should ask PID experts on if run 2 PIDCalib samples still lack low-PT Mu samples.

yipengsun commented 2 years ago
For the Mu_nopt sample (this has only a total of 28 samples, compared to the 400+ samples for the Mu): y \ x 1 (4500.0) 2 (7000.0) 3 (9000.0) 4 (11000.0) 5 (13250.0) 6 (16000.0) 7 (19500.0) 8 (24250.0) 9 (29500.0) 10 (36000.0) 11 (50000.0) 12 (65000.0) 13 (85000.0)
1 (1.9) 0.74 0.77 0.81 0.83 0.8 0.82 0.88 0.92 0.95 0.96 0.96 0.95 0.92
2 (2.8) 0.75 0.9 0.87 0.85 0.84 0.85 0.88 0.92 0.95 0.96 0.97 0.97 0.96
3 (3.7) 0.94 0.9 0.84 0.8 0.78 0.76 0.76 0.79 0.82 0.86 0.9 0.93 0.94
4 (4.6) 0.79 0.84 0.87 0.85 0.85 0.76 0.71 0.69 0.68 0.7 0.72 0.7 0.66
yipengsun commented 2 years ago

For 3D binning on the no PT cut sample:

[I 211007 02:58:25 make_eff_hists:179] Running PIDCalib2 make_eff_hists with the following config:
[I 211007 02:58:25 utils:116] ===============
[I 211007 02:58:25 utils:119] sample         : Turbo16
[I 211007 02:58:25 utils:119] magnet         : down
[I 211007 02:58:25 utils:119] particle       : Mu_nopt
[I 211007 02:58:25 utils:119] pid_cuts       : ['DLLmu>2.0&DLLe<1.0&IsMuon==1.0']
[I 211007 02:58:25 utils:119] bin_vars       : ['P', 'ETA', 'nTracks']
[I 211007 02:58:25 utils:119] output_dir     : pidcalib_output
[I 211007 02:58:25 utils:119] verbose        : False
[I 211007 02:58:25 utils:119] version        : 1.0.1
[I 211007 02:58:25 utils:120] ===============
[I 211007 02:58:25 binning:185] No 'P' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211007 02:58:25 binning:185] No 'ETA' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211007 02:58:25 binning:185] No 'nTracks' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211007 02:58:25 utils:450] Branches to be read: {'sWeight': 'probe_sWeight', 'DLLmu': 'probe_PIDmu', 'DLLe': 'probe_PIDe', 'IsMuon': 'probe_isMuon', 'P': 'probe_P', 'ETA': 'probe_ETA', 'nTracks': 'nTracks'}
[I 211007 02:58:25 utils:451] 28 calibration files from EOS will be processed
[I 211007 02:58:25 binning:185] No 'P' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211007 02:58:25 binning:185] No 'ETA' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211007 02:58:25 binning:185] No 'nTracks' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211007 03:03:15 utils:501] Processed 28/28 files                                                                                                                                                                                           
[I 211007 03:03:15 utils:580] 58340453/60339195 (96.7%) events passed binning range cut
[I 211007 03:03:15 utils:580] 20173532/58340453 (34.6%) events passed 'DLLmu>2.0&DLLe<1.0&IsMuon==1.0' cut
[W 211007 03:03:15 utils:62] There are 55 empty bins in the total histogram! You might want to change the binning.
[W 211007 03:03:15 utils:96] There are 3 negative bins in the 'eff_DLLmu>2.0&DLLe<1.0&IsMuon==1.0' efficiency histogram! You might want to change the binning.
[I 211007 03:03:15 make_eff_hists:215] Efficiency histograms saved to 'pidcalib_output/effhists-Turbo16-down-Mu_nopt-DLLmu>2.0&DLLe<1.0&IsMuon==1.0-P.ETA.nTracks.pkl'

File: run2-rdx-2016-md-Mu_nopt-p_eta_ntracks.root, Histo: eff Slicing axis: Y

Y, 1 (1.9)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (4500.0) 0.74 ± 0.00 0.74 ± 0.00 0.67 ± 0.22 nan ± nan
2 (7000.0) 0.77 ± 0.00 0.77 ± 0.00 1.20 ± 0.45 nan ± nan
3 (9000.0) 0.81 ± 0.00 0.81 ± 0.00 0.54 ± 0.27 nan ± nan
4 (11000.0) 0.83 ± 0.00 0.83 ± 0.00 1.00 ± 0.00 nan ± nan
5 (13250.0) 0.80 ± 0.00 0.80 ± 0.00 1.00 ± 0.00 nan ± nan
6 (16000.0) 0.82 ± 0.00 0.82 ± 0.00 0.56 ± 0.35 nan ± nan
7 (19500.0) 0.87 ± 0.00 0.88 ± 0.00 1.00 ± 0.00 nan ± nan
8 (24250.0) 0.92 ± 0.00 0.93 ± 0.00 1.00 ± 0.00 nan ± nan
9 (29500.0) 0.95 ± 0.00 0.95 ± 0.00 nan ± nan nan ± nan
10 (36000.0) 0.96 ± 0.00 0.95 ± 0.00 0.74 ± 0.69 nan ± nan
11 (50000.0) 0.96 ± 0.00 0.96 ± 0.00 -0.00 ± 0.00 nan ± nan
12 (65000.0) 0.96 ± 0.01 0.95 ± 0.01 nan ± nan nan ± nan
13 (85000.0) 0.93 ± 0.01 0.92 ± 0.01 nan ± nan nan ± nan

Y, 2 (2.8)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (4500.0) 0.76 ± 0.00 0.74 ± 0.00 3.65 ± 13.31 nan ± nan
2 (7000.0) 0.91 ± 0.00 0.88 ± 0.00 0.56 ± 0.33 nan ± nan
3 (9000.0) 0.88 ± 0.00 0.85 ± 0.00 0.70 ± 0.27 nan ± nan
4 (11000.0) 0.86 ± 0.00 0.84 ± 0.00 0.44 ± 0.28 nan ± nan
5 (13250.0) 0.85 ± 0.00 0.83 ± 0.00 2.01 ± 5.68 nan ± nan
6 (16000.0) 0.86 ± 0.00 0.84 ± 0.00 0.76 ± 0.22 nan ± nan
7 (19500.0) 0.88 ± 0.00 0.87 ± 0.00 0.66 ± 0.49 nan ± nan
8 (24250.0) 0.93 ± 0.00 0.92 ± 0.00 1.00 ± 0.31 nan ± nan
9 (29500.0) 0.95 ± 0.00 0.94 ± 0.00 0.72 ± 0.33 nan ± nan
10 (36000.0) 0.96 ± 0.00 0.96 ± 0.00 1.32 ± 0.51 nan ± nan
11 (50000.0) 0.97 ± 0.00 0.97 ± 0.00 1.17 ± 0.17 nan ± nan
12 (65000.0) 0.97 ± 0.00 0.96 ± 0.00 0.00 ± 0.00 nan ± nan
13 (85000.0) 0.96 ± 0.00 0.96 ± 0.00 1.00 ± 0.00 nan ± nan

Y, 3 (3.7)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (4500.0) 0.97 ± 0.01 0.92 ± 0.02 0.00 ± 0.00 nan ± nan
2 (7000.0) 0.92 ± 0.01 0.87 ± 0.01 -0.14 ± 0.21 nan ± nan
3 (9000.0) 0.86 ± 0.00 0.82 ± 0.00 0.53 ± 0.89 nan ± nan
4 (11000.0) 0.82 ± 0.00 0.78 ± 0.00 3.45 ± 37.08 nan ± nan
5 (13250.0) 0.79 ± 0.00 0.77 ± 0.00 0.15 ± 0.17 nan ± nan
6 (16000.0) 0.77 ± 0.00 0.74 ± 0.00 1.09 ± 1.06 nan ± nan
7 (19500.0) 0.77 ± 0.00 0.74 ± 0.00 0.34 ± 0.25 nan ± nan
8 (24250.0) 0.81 ± 0.00 0.78 ± 0.00 0.62 ± 0.20 nan ± nan
9 (29500.0) 0.84 ± 0.00 0.81 ± 0.00 0.83 ± 0.20 nan ± nan
10 (36000.0) 0.87 ± 0.00 0.85 ± 0.00 0.52 ± 0.32 nan ± nan
11 (50000.0) 0.90 ± 0.00 0.89 ± 0.00 5.04 ± 14.07 nan ± nan
12 (65000.0) 0.93 ± 0.00 0.92 ± 0.00 1.35 ± 0.46 nan ± nan
13 (85000.0) 0.94 ± 0.00 0.93 ± 0.00 2.35 ± 2.30 nan ± nan

Y, 4 (4.6)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (4500.0) 0.87 ± 0.14 0.72 ± 0.19 0.00 ± 0.00 nan ± nan
2 (7000.0) 0.91 ± 0.06 0.77 ± 0.08 -0.00 ± 0.00 nan ± nan
3 (9000.0) 0.89 ± 0.03 0.84 ± 0.05 -0.00 ± 0.00 nan ± nan
4 (11000.0) 0.88 ± 0.02 0.82 ± 0.03 0.00 ± 0.00 nan ± nan
5 (13250.0) 0.87 ± 0.01 0.83 ± 0.02 -0.00 ± 0.00 nan ± nan
6 (16000.0) 0.81 ± 0.01 0.72 ± 0.01 -0.00 ± 0.00 nan ± nan
7 (19500.0) 0.75 ± 0.01 0.67 ± 0.01 0.00 ± 0.00 nan ± nan
8 (24250.0) 0.72 ± 0.00 0.65 ± 0.01 0.25 ± 0.23 nan ± nan
9 (29500.0) 0.72 ± 0.00 0.64 ± 0.00 -0.00 ± 0.00 nan ± nan
10 (36000.0) 0.73 ± 0.00 0.66 ± 0.00 -2.78 ± 25.93 nan ± nan
11 (50000.0) 0.74 ± 0.00 0.69 ± 0.00 -0.30 ± 1.91 nan ± nan
12 (65000.0) 0.72 ± 0.00 0.69 ± 0.00 -0.00 ± 0.00 nan ± nan
13 (85000.0) 0.67 ± 0.00 0.65 ± 0.00 0.90 ± 0.35 nan ± nan
yipengsun commented 2 years ago

For 3D binning on the regular Mu sample:

[I 211012 04:15:39 make_eff_hists:179] Running PIDCalib2 make_eff_hists with the following config:
[I 211012 04:15:39 utils:116] ===============
[I 211012 04:15:39 utils:119] sample         : Turbo16
[I 211012 04:15:39 utils:119] magnet         : down
[I 211012 04:15:39 utils:119] particle       : Mu
[I 211012 04:15:39 utils:119] pid_cuts       : ['DLLmu>2.0&DLLe<1.0&IsMuon==1.0']
[I 211012 04:15:39 utils:119] bin_vars       : ['P', 'ETA', 'nTracks']
[I 211012 04:15:39 utils:119] output_dir     : pidcalib_output
[I 211012 04:15:39 utils:119] verbose        : False
[I 211012 04:15:39 utils:119] version        : 1.0.1
[I 211012 04:15:39 utils:120] ===============
[I 211012 04:15:39 utils:450] Branches to be read: {'sWeight': 'probe_sWeight', 'DLLmu': 'probe_PIDmu', 'DLLe': 'probe_PIDe', 'IsMuon': 'probe_isMuon', 'P': 'probe_P', 'ETA': 'probe_ETA', 'nTracks': 'nTracks'}
[I 211012 04:15:39 utils:451] 258 calibration files from EOS will be processed
[I 211012 04:24:50 utils:501] Processed 258/258 files                                                                                                                                                                                         
[I 211012 04:24:50 utils:580] 46137361/47995738 (96.1%) events passed binning range cut
[I 211012 04:24:50 utils:580] 18756855/46137361 (40.7%) events passed 'DLLmu>2.0&DLLe<1.0&IsMuon==1.0' cut
[W 211012 04:24:50 utils:62] There are 86 empty bins in the total histogram! You might want to change the binning.
[W 211012 04:24:50 utils:96] There are 4 negative bins in the 'eff_DLLmu>2.0&DLLe<1.0&IsMuon==1.0' efficiency histogram! You might want to change the binning.
[I 211012 04:24:50 make_eff_hists:215] Efficiency histograms saved to 'pidcalib_output/effhists-Turbo16-down-Mu-DLLmu>2.0&DLLe<1.0&IsMuon==1.0-P.ETA.nTracks.pkl'

File: run2-rdx-2016-md-Mu-p_eta_ntracks.root, Histo: eff Slicing axis: Y

Y, 1 (1.9)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (4500.0) 0.77 ± 0.00 0.76 ± 0.00 0.96 ± 0.35 nan ± nan
2 (7000.0) 0.77 ± 0.00 0.77 ± 0.00 1.10 ± 0.25 nan ± nan
3 (9000.0) 0.81 ± 0.00 0.81 ± 0.00 0.53 ± 0.27 nan ± nan
4 (11000.0) 0.83 ± 0.00 0.83 ± 0.00 1.11 ± 0.13 nan ± nan
5 (13250.0) 0.80 ± 0.00 0.80 ± 0.00 1.00 ± 0.00 nan ± nan
6 (16000.0) 0.82 ± 0.00 0.82 ± 0.00 0.55 ± 0.35 nan ± nan
7 (19500.0) 0.87 ± 0.00 0.88 ± 0.00 1.00 ± 0.00 nan ± nan
8 (24250.0) 0.92 ± 0.00 0.93 ± 0.00 1.00 ± 0.00 nan ± nan
9 (29500.0) 0.95 ± 0.00 0.95 ± 0.00 nan ± nan nan ± nan
10 (36000.0) 0.96 ± 0.00 0.95 ± 0.00 0.76 ± 0.71 nan ± nan
11 (50000.0) 0.97 ± 0.00 0.96 ± 0.00 -0.00 ± 0.00 nan ± nan
12 (65000.0) 0.96 ± 0.01 0.95 ± 0.01 nan ± nan nan ± nan
13 (85000.0) 0.93 ± 0.01 0.91 ± 0.01 nan ± nan nan ± nan

Y, 2 (2.8)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (4500.0) 0.78 ± 0.00 0.76 ± 0.00 -0.55 ± 0.90 nan ± nan
2 (7000.0) 0.91 ± 0.00 0.89 ± 0.00 0.70 ± 0.37 nan ± nan
3 (9000.0) 0.88 ± 0.00 0.85 ± 0.00 0.85 ± 0.28 nan ± nan
4 (11000.0) 0.86 ± 0.00 0.84 ± 0.00 0.49 ± 0.32 nan ± nan
5 (13250.0) 0.85 ± 0.00 0.83 ± 0.00 2.06 ± 6.00 nan ± nan
6 (16000.0) 0.86 ± 0.00 0.84 ± 0.00 0.77 ± 0.22 nan ± nan
7 (19500.0) 0.88 ± 0.00 0.87 ± 0.00 0.92 ± 0.90 nan ± nan
8 (24250.0) 0.93 ± 0.00 0.92 ± 0.00 1.07 ± 0.30 nan ± nan
9 (29500.0) 0.95 ± 0.00 0.94 ± 0.00 0.72 ± 0.33 nan ± nan
10 (36000.0) 0.97 ± 0.00 0.96 ± 0.00 1.30 ± 0.48 nan ± nan
11 (50000.0) 0.97 ± 0.00 0.97 ± 0.00 0.92 ± 0.22 nan ± nan
12 (65000.0) 0.97 ± 0.00 0.97 ± 0.00 0.00 ± 0.00 nan ± nan
13 (85000.0) 0.96 ± 0.00 0.96 ± 0.00 1.00 ± 0.00 nan ± nan

Y, 3 (3.7)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (4500.0) nan ± nan nan ± nan nan ± nan nan ± nan
2 (7000.0) nan ± nan nan ± nan nan ± nan nan ± nan
3 (9000.0) nan ± nan nan ± nan nan ± nan nan ± nan
4 (11000.0) 0.81 ± 0.00 0.78 ± 0.01 -0.00 ± 0.00 nan ± nan
5 (13250.0) 0.79 ± 0.00 0.77 ± 0.00 2.63 ± 32.62 nan ± nan
6 (16000.0) 0.77 ± 0.00 0.75 ± 0.00 -9.10 ± 84.40 nan ± nan
7 (19500.0) 0.78 ± 0.00 0.75 ± 0.00 0.37 ± 0.31 nan ± nan
8 (24250.0) 0.81 ± 0.00 0.78 ± 0.00 0.61 ± 0.20 nan ± nan
9 (29500.0) 0.84 ± 0.00 0.81 ± 0.00 0.86 ± 0.22 nan ± nan
10 (36000.0) 0.87 ± 0.00 0.85 ± 0.00 0.59 ± 0.38 nan ± nan
11 (50000.0) 0.90 ± 0.00 0.89 ± 0.00 2.12 ± 3.06 nan ± nan
12 (65000.0) 0.93 ± 0.00 0.92 ± 0.00 1.34 ± 0.44 nan ± nan
13 (85000.0) 0.95 ± 0.00 0.93 ± 0.00 2.39 ± 2.43 nan ± nan

Y, 4 (4.6)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (4500.0) nan ± nan nan ± nan nan ± nan nan ± nan
2 (7000.0) nan ± nan nan ± nan nan ± nan nan ± nan
3 (9000.0) nan ± nan nan ± nan nan ± nan nan ± nan
4 (11000.0) nan ± nan nan ± nan nan ± nan nan ± nan
5 (13250.0) nan ± nan nan ± nan nan ± nan nan ± nan
6 (16000.0) nan ± nan nan ± nan nan ± nan nan ± nan
7 (19500.0) nan ± nan nan ± nan nan ± nan nan ± nan
8 (24250.0) 0.71 ± 0.01 0.70 ± 0.02 nan ± nan nan ± nan
9 (29500.0) 0.73 ± 0.00 0.68 ± 0.01 -0.00 ± 0.00 nan ± nan
10 (36000.0) 0.75 ± 0.00 0.69 ± 0.00 -1.43 ± 5.76 nan ± nan
11 (50000.0) 0.74 ± 0.00 0.70 ± 0.00 -0.30 ± 1.95 nan ± nan
12 (65000.0) 0.71 ± 0.00 0.69 ± 0.00 -0.00 ± 0.00 nan ± nan
13 (85000.0) 0.67 ± 0.00 0.65 ± 0.00 0.90 ± 0.52 nan ± nan
yipengsun commented 2 years ago

Looking at K PID weights w/ 3D binning:

File: run2-rdx-2015-md-K-p_eta_ntracks.root, Histo: eff Slicing axis: Y

Y, 1 (1.9)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (6150.0) 0.96 ± 0.00 0.95 ± 0.00 0.92 ± 0.08 nan ± nan
2 (12450.0) 0.97 ± 0.00 0.97 ± 0.00 1.00 ± 0.00 nan ± nan
3 (17300.0) 0.98 ± 0.00 0.97 ± 0.00 1.00 ± 0.00 nan ± nan
4 (21700.0) 0.98 ± 0.00 0.98 ± 0.00 1.00 ± 0.00 nan ± nan
5 (27100.0) 0.98 ± 0.00 0.97 ± 0.00 1.00 ± 0.00 nan ± nan
6 (32500.0) 0.95 ± 0.00 0.94 ± 0.00 nan ± nan nan ± nan
7 (37900.0) 0.88 ± 0.00 0.85 ± 0.00 1.00 ± 0.00 nan ± nan
8 (43300.0) 0.76 ± 0.00 0.73 ± 0.00 0.00 ± 0.00 nan ± nan
9 (48700.0) 0.61 ± 0.00 0.58 ± 0.00 nan ± nan nan ± nan
10 (54100.0) 0.45 ± 0.01 0.42 ± 0.01 nan ± nan nan ± nan
11 (59500.0) 0.30 ± 0.01 0.27 ± 0.01 nan ± nan nan ± nan
12 (64900.0) 0.19 ± 0.01 0.18 ± 0.01 nan ± nan nan ± nan
13 (70300.0) 0.13 ± 0.01 0.11 ± 0.01 nan ± nan nan ± nan
14 (75700.0) 0.08 ± 0.01 0.05 ± 0.01 nan ± nan nan ± nan
15 (81100.0) 0.03 ± 0.01 0.03 ± 0.01 nan ± nan nan ± nan
16 (86500.0) 0.02 ± 0.00 0.01 ± 0.00 nan ± nan nan ± nan
17 (91900.0) 0.01 ± 0.00 0.01 ± 0.00 nan ± nan nan ± nan
18 (97300.0) 0.01 ± 0.00 0.01 ± 0.00 nan ± nan nan ± nan

Y, 2 (2.8)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (6150.0) 0.90 ± 0.00 0.85 ± 0.00 1.18 ± 1.14 nan ± nan
2 (12450.0) 0.96 ± 0.00 0.94 ± 0.00 0.60 ± 0.17 nan ± nan
3 (17300.0) 0.97 ± 0.00 0.97 ± 0.00 0.58 ± 0.36 nan ± nan
4 (21700.0) 0.97 ± 0.00 0.97 ± 0.00 0.88 ± 0.16 nan ± nan
5 (27100.0) 0.97 ± 0.00 0.97 ± 0.00 1.40 ± 0.50 nan ± nan
6 (32500.0) 0.97 ± 0.00 0.95 ± 0.00 0.75 ± 0.22 nan ± nan
7 (37900.0) 0.94 ± 0.00 0.93 ± 0.00 1.00 ± 0.00 nan ± nan
8 (43300.0) 0.92 ± 0.00 0.90 ± 0.00 0.00 ± 0.00 nan ± nan
9 (48700.0) 0.90 ± 0.00 0.88 ± 0.00 1.00 ± 0.00 nan ± nan
10 (54100.0) 0.88 ± 0.00 0.85 ± 0.00 1.00 ± 0.00 nan ± nan
11 (59500.0) 0.85 ± 0.00 0.81 ± 0.00 nan ± nan nan ± nan
12 (64900.0) 0.81 ± 0.00 0.77 ± 0.00 nan ± nan nan ± nan
13 (70300.0) 0.76 ± 0.00 0.71 ± 0.00 nan ± nan nan ± nan
14 (75700.0) 0.70 ± 0.00 0.65 ± 0.00 0.00 ± 0.00 nan ± nan
15 (81100.0) 0.63 ± 0.00 0.57 ± 0.00 nan ± nan nan ± nan
16 (86500.0) 0.55 ± 0.00 0.50 ± 0.00 nan ± nan nan ± nan
17 (91900.0) 0.49 ± 0.00 0.43 ± 0.00 nan ± nan nan ± nan
18 (97300.0) 0.42 ± 0.00 0.36 ± 0.00 0.00 ± 0.00 nan ± nan

Y, 3 (3.7)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (6150.0) 0.82 ± 0.00 0.72 ± 0.00 22.51 ± 1046.13 nan ± nan
2 (12450.0) 0.94 ± 0.00 0.89 ± 0.00 0.86 ± 0.16 nan ± nan
3 (17300.0) 0.96 ± 0.00 0.93 ± 0.00 1.00 ± 0.00 nan ± nan
4 (21700.0) 0.97 ± 0.00 0.94 ± 0.00 1.15 ± 0.57 nan ± nan
5 (27100.0) 0.97 ± 0.00 0.95 ± 0.00 0.75 ± 0.34 nan ± nan
6 (32500.0) 0.96 ± 0.00 0.94 ± 0.00 0.26 ± 0.51 nan ± nan
7 (37900.0) 0.96 ± 0.00 0.94 ± 0.00 1.35 ± 0.66 nan ± nan
8 (43300.0) 0.95 ± 0.00 0.93 ± 0.00 0.89 ± 0.10 nan ± nan
9 (48700.0) 0.94 ± 0.00 0.91 ± 0.00 0.71 ± 0.23 nan ± nan
10 (54100.0) 0.92 ± 0.00 0.88 ± 0.00 -0.00 ± 0.00 nan ± nan
11 (59500.0) 0.89 ± 0.00 0.84 ± 0.00 -4.25 ± 29.78 nan ± nan
12 (64900.0) 0.84 ± 0.00 0.78 ± 0.00 1.00 ± 0.00 nan ± nan
13 (70300.0) 0.78 ± 0.00 0.71 ± 0.00 nan ± nan nan ± nan
14 (75700.0) 0.72 ± 0.00 0.64 ± 0.00 0.44 ± 0.48 nan ± nan
15 (81100.0) 0.63 ± 0.00 0.55 ± 0.00 0.08 ± 0.11 nan ± nan
16 (86500.0) 0.55 ± 0.00 0.47 ± 0.00 nan ± nan nan ± nan
17 (91900.0) 0.46 ± 0.00 0.39 ± 0.00 nan ± nan nan ± nan
18 (97300.0) 0.39 ± 0.00 0.32 ± 0.00 0.00 ± 0.00 nan ± nan

Y, 4 (4.6)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (6150.0) 0.25 ± 0.02 0.08 ± 0.02 nan ± nan nan ± nan
2 (12450.0) 0.73 ± 0.01 0.55 ± 0.01 -0.00 ± 0.00 nan ± nan
3 (17300.0) 0.84 ± 0.01 0.69 ± 0.01 1.00 ± 0.00 nan ± nan
4 (21700.0) 0.86 ± 0.00 0.77 ± 0.00 0.83 ± 0.20 nan ± nan
5 (27100.0) 0.85 ± 0.00 0.79 ± 0.00 -0.00 ± 0.00 nan ± nan
6 (32500.0) 0.82 ± 0.00 0.77 ± 0.00 0.00 ± 0.00 nan ± nan
7 (37900.0) 0.80 ± 0.00 0.75 ± 0.00 1.00 ± 0.00 nan ± nan
8 (43300.0) 0.80 ± 0.00 0.76 ± 0.00 1.56 ± 1.07 nan ± nan
9 (48700.0) 0.79 ± 0.00 0.75 ± 0.00 1.00 ± 0.00 nan ± nan
10 (54100.0) 0.77 ± 0.00 0.72 ± 0.00 nan ± nan nan ± nan
11 (59500.0) 0.72 ± 0.00 0.67 ± 0.00 0.89 ± 0.12 nan ± nan
12 (64900.0) 0.68 ± 0.00 0.62 ± 0.00 1.00 ± 0.00 nan ± nan
13 (70300.0) 0.60 ± 0.00 0.55 ± 0.00 0.00 ± 0.00 nan ± nan
14 (75700.0) 0.53 ± 0.00 0.47 ± 0.00 nan ± nan nan ± nan
15 (81100.0) 0.46 ± 0.00 0.40 ± 0.00 0.00 ± 0.00 nan ± nan
16 (86500.0) 0.38 ± 0.00 0.32 ± 0.00 0.00 ± 0.00 nan ± nan
17 (91900.0) 0.31 ± 0.00 0.27 ± 0.00 nan ± nan nan ± nan
18 (97300.0) 0.25 ± 0.00 0.21 ± 0.00 nan ± nan nan ± nan
yipengsun commented 2 years ago

2016 is not getting any better. The larger nTracks bins are often unfilled:

File: run2-rdx-2016-md-K-p_eta_ntracks.root, Histo: eff Slicing axis: Y

Y, 1 (1.9)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (6150.0) 0.95 ± 0.00 0.95 ± 0.00 0.84 ± 0.06 nan ± nan
2 (12450.0) 0.96 ± 0.00 0.96 ± 0.00 0.94 ± 0.03 nan ± nan
3 (17300.0) 0.97 ± 0.00 0.97 ± 0.00 1.06 ± 0.05 nan ± nan
4 (21700.0) 0.98 ± 0.00 0.97 ± 0.00 0.98 ± 0.02 nan ± nan
5 (27100.0) 0.98 ± 0.00 0.97 ± 0.00 0.98 ± 0.02 nan ± nan
6 (32500.0) 0.94 ± 0.00 0.93 ± 0.00 1.18 ± 0.19 nan ± nan
7 (37900.0) 0.86 ± 0.00 0.83 ± 0.00 0.40 ± 0.33 nan ± nan
8 (43300.0) 0.74 ± 0.00 0.70 ± 0.00 0.75 ± 0.15 nan ± nan
9 (48700.0) 0.63 ± 0.00 0.59 ± 0.00 0.67 ± 0.21 nan ± nan
10 (54100.0) 0.51 ± 0.00 0.47 ± 0.00 0.45 ± 0.35 nan ± nan
11 (59500.0) 0.40 ± 0.00 0.36 ± 0.00 0.00 ± 0.00 nan ± nan
12 (64900.0) 0.30 ± 0.00 0.27 ± 0.00 0.00 ± 0.00 nan ± nan
13 (70300.0) 0.22 ± 0.00 0.19 ± 0.00 nan ± nan nan ± nan
14 (75700.0) 0.15 ± 0.00 0.13 ± 0.00 nan ± nan nan ± nan
15 (81100.0) 0.10 ± 0.00 0.09 ± 0.00 0.00 ± 0.00 nan ± nan
16 (86500.0) 0.07 ± 0.00 0.05 ± 0.00 nan ± nan nan ± nan
17 (91900.0) 0.04 ± 0.00 0.03 ± 0.00 nan ± nan nan ± nan
18 (97300.0) 0.02 ± 0.00 0.02 ± 0.00 nan ± nan nan ± nan

Y, 2 (2.8)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (6150.0) 0.89 ± 0.00 0.84 ± 0.00 0.68 ± 0.08 nan ± nan
2 (12450.0) 0.95 ± 0.00 0.94 ± 0.00 0.88 ± 0.04 nan ± nan
3 (17300.0) 0.96 ± 0.00 0.96 ± 0.00 0.90 ± 0.04 nan ± nan
4 (21700.0) 0.97 ± 0.00 0.97 ± 0.00 0.90 ± 0.03 nan ± nan
5 (27100.0) 0.97 ± 0.00 0.97 ± 0.00 0.82 ± 0.04 nan ± nan
6 (32500.0) 0.96 ± 0.00 0.95 ± 0.00 0.98 ± 0.04 nan ± nan
7 (37900.0) 0.94 ± 0.00 0.93 ± 0.00 0.90 ± 0.03 nan ± nan
8 (43300.0) 0.92 ± 0.00 0.90 ± 0.00 0.84 ± 0.05 nan ± nan
9 (48700.0) 0.89 ± 0.00 0.87 ± 0.00 0.87 ± 0.08 nan ± nan
10 (54100.0) 0.87 ± 0.00 0.84 ± 0.00 0.77 ± 0.10 nan ± nan
11 (59500.0) 0.84 ± 0.00 0.81 ± 0.00 0.57 ± 0.14 nan ± nan
12 (64900.0) 0.80 ± 0.00 0.76 ± 0.00 0.45 ± 0.14 nan ± nan
13 (70300.0) 0.75 ± 0.00 0.70 ± 0.00 0.54 ± 0.12 nan ± nan
14 (75700.0) 0.69 ± 0.00 0.63 ± 0.00 0.54 ± 0.17 nan ± nan
15 (81100.0) 0.62 ± 0.00 0.56 ± 0.00 0.27 ± 0.17 nan ± nan
16 (86500.0) 0.54 ± 0.00 0.48 ± 0.00 -0.10 ± 1.07 nan ± nan
17 (91900.0) 0.47 ± 0.00 0.41 ± 0.00 0.00 ± 0.00 nan ± nan
18 (97300.0) 0.40 ± 0.00 0.34 ± 0.00 0.48 ± 0.23 nan ± nan

Y, 3 (3.7)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (6150.0) 0.82 ± 0.00 0.72 ± 0.00 0.78 ± 0.17 nan ± nan
2 (12450.0) 0.94 ± 0.00 0.90 ± 0.00 0.82 ± 0.10 nan ± nan
3 (17300.0) 0.96 ± 0.00 0.93 ± 0.00 0.83 ± 0.14 nan ± nan
4 (21700.0) 0.96 ± 0.00 0.95 ± 0.00 0.88 ± 0.07 nan ± nan
5 (27100.0) 0.96 ± 0.00 0.95 ± 0.00 0.86 ± 0.05 nan ± nan
6 (32500.0) 0.96 ± 0.00 0.94 ± 0.00 0.86 ± 0.07 nan ± nan
7 (37900.0) 0.95 ± 0.00 0.93 ± 0.00 0.83 ± 0.07 nan ± nan
8 (43300.0) 0.95 ± 0.00 0.93 ± 0.00 0.78 ± 0.06 nan ± nan
9 (48700.0) 0.93 ± 0.00 0.91 ± 0.00 0.70 ± 0.06 nan ± nan
10 (54100.0) 0.91 ± 0.00 0.87 ± 0.00 0.63 ± 0.06 nan ± nan
11 (59500.0) 0.87 ± 0.00 0.83 ± 0.00 0.86 ± 0.10 nan ± nan
12 (64900.0) 0.82 ± 0.00 0.77 ± 0.00 0.60 ± 0.07 nan ± nan
13 (70300.0) 0.76 ± 0.00 0.69 ± 0.00 0.57 ± 0.11 nan ± nan
14 (75700.0) 0.68 ± 0.00 0.61 ± 0.00 0.70 ± 0.17 nan ± nan
15 (81100.0) 0.60 ± 0.00 0.53 ± 0.00 0.21 ± 0.10 nan ± nan
16 (86500.0) 0.52 ± 0.00 0.45 ± 0.00 0.31 ± 0.12 nan ± nan
17 (91900.0) 0.44 ± 0.00 0.37 ± 0.00 0.17 ± 0.15 nan ± nan
18 (97300.0) 0.36 ± 0.00 0.30 ± 0.00 0.05 ± 0.09 nan ± nan

Y, 4 (4.6)

X \ Z 1 (25.0) 2 (125.0) 3 (250.0) 4 (400.0)
1 (6150.0) 0.27 ± 0.01 0.09 ± 0.01 -1.52 ± 4.64 nan ± nan
2 (12450.0) 0.72 ± 0.00 0.56 ± 0.00 -0.57 ± 6.69 nan ± nan
3 (17300.0) 0.83 ± 0.00 0.69 ± 0.00 0.05 ± 0.78 nan ± nan
4 (21700.0) 0.86 ± 0.00 0.78 ± 0.00 1.97 ± 5.56 nan ± nan
5 (27100.0) 0.85 ± 0.00 0.79 ± 0.00 -0.02 ± 0.56 nan ± nan
6 (32500.0) 0.82 ± 0.00 0.77 ± 0.00 1.04 ± 0.79 nan ± nan
7 (37900.0) 0.80 ± 0.00 0.75 ± 0.00 0.64 ± 0.23 nan ± nan
8 (43300.0) 0.79 ± 0.00 0.75 ± 0.00 0.37 ± 0.19 nan ± nan
9 (48700.0) 0.79 ± 0.00 0.74 ± 0.00 -5.85 ± 107.04 nan ± nan
10 (54100.0) 0.76 ± 0.00 0.72 ± 0.00 0.47 ± 0.22 nan ± nan
11 (59500.0) 0.72 ± 0.00 0.67 ± 0.00 0.44 ± 0.15 nan ± nan
12 (64900.0) 0.66 ± 0.00 0.61 ± 0.00 0.43 ± 0.16 nan ± nan
13 (70300.0) 0.60 ± 0.00 0.55 ± 0.00 0.58 ± 0.22 nan ± nan
14 (75700.0) 0.53 ± 0.00 0.48 ± 0.00 0.55 ± 0.25 nan ± nan
15 (81100.0) 0.46 ± 0.00 0.40 ± 0.00 0.22 ± 0.13 nan ± nan
16 (86500.0) 0.38 ± 0.00 0.33 ± 0.00 0.43 ± 0.24 nan ± nan
17 (91900.0) 0.31 ± 0.00 0.27 ± 0.00 0.01 ± 0.18 nan ± nan
18 (97300.0) 0.26 ± 0.00 0.22 ± 0.00 0.54 ± 0.32 nan ± nan
manuelfs commented 2 years ago

In this thread they tell us that we should not use nTracks in Run 2, which has a bug, and use instead nTracks_Brunel https://groups.cern.ch/group/lhcb-phys-pid-calibration/Lists/Archive/Flat.aspx?RootFolder=%2Fgroup%2Flhcb%2Dphys%2Dpid%2Dcalibration%2FLists%2FArchive%2Fpidcalib2%20sample%20P%2DETA%2DnTracks%20binning%20problem&FolderCTID=0x01200200428956A5A0587D4CA9DBE3C7098FA903

The point to this twiki that gives more details (they actually have a broken link in their email) https://twiki.cern.ch/twiki/bin/view/LHCb/PIDCalibPackage#Binning_variables_in_Run_1_and_R

manuelfs commented 2 years ago

Note that since our analysis is not Turbo, we should also use Brunel_P and Brunel_ETA

yipengsun commented 2 years ago

Indeed, when we use ALL offline variables (P, ETA, nTracks ALL offline), there's much fewer unfilled bins:

❯ ./run2-rdx_demo.sh 
[I 211015 23:44:28 make_eff_hists:179] Running PIDCalib2 make_eff_hists with the following config:
[I 211015 23:44:28 utils:116] ===============
[I 211015 23:44:28 utils:119] sample         : Turbo16
[I 211015 23:44:28 utils:119] magnet         : down
[I 211015 23:44:28 utils:119] particle       : Mu
[I 211015 23:44:28 utils:119] pid_cuts       : ['DLLmu>2.0&DLLe<1.0&IsMuon==1.0']
[I 211015 23:44:28 utils:119] bin_vars       : ['Brunel_P', 'Brunel_ETA', 'nTracks_Brunel']
[I 211015 23:44:28 utils:119] output_dir     : pidcalib_output
[I 211015 23:44:28 utils:119] verbose        : False
[I 211015 23:44:28 utils:119] version        : 1.0.1
[I 211015 23:44:28 utils:120] ===============
[I 211015 23:44:28 utils:450] Branches to be read: {'sWeight': 'probe_sWeight', 'DLLmu': 'probe_PIDmu', 'DLLe': 'probe_PIDe', 'IsMuon': 'probe_isMuon', 'Brunel_P': 'probe_Brunel_P', 'Brunel_ETA': 'probe_Brunel_ETA', 'nTracks_Brunel': 'nTracks_Brunel'}
[I 211015 23:44:28 utils:451] 258 calibration files from EOS will be processed
[I 211015 23:53:35 utils:501] Processed 258/258 files                                                                 
[I 211015 23:53:35 utils:580] 45074506/47995738 (93.9%) events passed binning range cut
[I 211015 23:53:35 utils:580] 18422113/45074506 (40.9%) events passed 'DLLmu>2.0&DLLe<1.0&IsMuon==1.0' cut
[W 211015 23:53:35 utils:62] There are 10 empty bins in the total histogram! You might want to change the binning.
[W 211015 23:53:35 utils:96] There are 6 negative bins in the 'eff_DLLmu>2.0&DLLe<1.0&IsMuon==1.0' efficiency histogram! You might want to change the binning.
[I 211015 23:53:35 make_eff_hists:215] Efficiency histograms saved to 'pidcalib_output/effhists-Turbo16-down-Mu-DLLmu>2.0&DLLe<1.0&IsMuon==1.0-Brunel_P.Brunel_ETA.nTracks_Brunel.pkl'
yipengsun commented 2 years ago

File: run2-rdx-2016-md-Mu-p_eta_ntracks_all_offline.root, Histo: eff Slicing axis: η

η, 1 (1.5)

p \ nTrk 1 (0.0) 2 (50.0) 3 (200.0) 4 (300.0)
1 (3000.0) 0.77 ± 0.00 0.77 ± 0.00 0.76 ± 0.00 0.75 ± 0.00
2 (6000.0) 0.78 ± 0.00 0.77 ± 0.00 0.77 ± 0.00 0.76 ± 0.00
3 (8000.0) 0.82 ± 0.01 0.81 ± 0.00 0.81 ± 0.00 0.80 ± 0.00
4 (10000.0) 0.84 ± 0.01 0.83 ± 0.00 0.83 ± 0.00 0.83 ± 0.00
5 (12000.0) 0.80 ± 0.01 0.80 ± 0.00 0.80 ± 0.00 0.81 ± 0.00
6 (14500.0) 0.81 ± 0.01 0.81 ± 0.00 0.82 ± 0.00 0.82 ± 0.00
7 (17500.0) 0.86 ± 0.01 0.88 ± 0.00 0.87 ± 0.00 0.88 ± 0.00
8 (21500.0) 0.92 ± 0.01 0.92 ± 0.00 0.92 ± 0.00 0.92 ± 0.00
9 (27000.0) 0.98 ± 0.01 0.95 ± 0.00 0.95 ± 0.00 0.95 ± 0.01
10 (32000.0) 0.96 ± 0.01 0.96 ± 0.00 0.96 ± 0.00 0.95 ± 0.01
11 (40000.0) 0.98 ± 0.02 0.96 ± 0.00 0.96 ± 0.00 0.97 ± 0.01
12 (60000.0) 0.94 ± 0.05 0.96 ± 0.01 0.95 ± 0.01 0.91 ± 0.01
13 (70000.0) 0.90 ± 0.04 0.94 ± 0.01 0.92 ± 0.01 0.89 ± 0.02

η, 2 (2.4)

p \ nTrk 1 (0.0) 2 (50.0) 3 (200.0) 4 (300.0)
1 (3000.0) 0.77 ± 0.01 0.77 ± 0.00 0.77 ± 0.00 0.74 ± 0.01
2 (6000.0) 0.93 ± 0.00 0.91 ± 0.00 0.89 ± 0.00 0.87 ± 0.00
3 (8000.0) 0.90 ± 0.00 0.88 ± 0.00 0.86 ± 0.00 0.84 ± 0.00
4 (10000.0) 0.87 ± 0.00 0.86 ± 0.00 0.85 ± 0.00 0.83 ± 0.00
5 (12000.0) 0.86 ± 0.00 0.85 ± 0.00 0.84 ± 0.00 0.82 ± 0.00
6 (14500.0) 0.87 ± 0.00 0.85 ± 0.00 0.84 ± 0.00 0.83 ± 0.00
7 (17500.0) 0.89 ± 0.00 0.88 ± 0.00 0.88 ± 0.00 0.87 ± 0.00
8 (21500.0) 0.93 ± 0.00 0.93 ± 0.00 0.92 ± 0.00 0.91 ± 0.00
9 (27000.0) 0.96 ± 0.00 0.95 ± 0.00 0.95 ± 0.00 0.94 ± 0.00
10 (32000.0) 0.97 ± 0.00 0.96 ± 0.00 0.96 ± 0.00 0.96 ± 0.00
11 (40000.0) 0.97 ± 0.00 0.97 ± 0.00 0.97 ± 0.00 0.96 ± 0.00
12 (60000.0) 0.97 ± 0.01 0.97 ± 0.00 0.96 ± 0.00 0.97 ± 0.00
13 (70000.0) 0.96 ± 0.01 0.96 ± 0.00 0.96 ± 0.00 0.95 ± 0.00

η, 3 (3.2)

p \ nTrk 1 (0.0) 2 (50.0) 3 (200.0) 4 (300.0)
1 (3000.0) nan 0.00 ± 0.00 -0.15 ± 0.66 1.07 ± 6.29
2 (6000.0) nan 0.00 ± 0.00 -3.64 ± 16.42 0.45 ± 0.60
3 (8000.0) nan -0.27 ± 1.01 0.80 ± 0.67 -0.19 ± 0.32
4 (10000.0) 0.82 ± 0.02 0.80 ± 0.00 0.78 ± 0.01 0.78 ± 0.01
5 (12000.0) 0.80 ± 0.01 0.79 ± 0.00 0.78 ± 0.00 0.75 ± 0.01
6 (14500.0) 0.79 ± 0.01 0.77 ± 0.00 0.74 ± 0.00 0.74 ± 0.00
7 (17500.0) 0.79 ± 0.00 0.77 ± 0.00 0.75 ± 0.00 0.73 ± 0.00
8 (21500.0) 0.82 ± 0.00 0.80 ± 0.00 0.78 ± 0.00 0.77 ± 0.00
9 (27000.0) 0.85 ± 0.00 0.83 ± 0.00 0.81 ± 0.00 0.79 ± 0.00
10 (32000.0) 0.87 ± 0.00 0.87 ± 0.00 0.85 ± 0.00 0.83 ± 0.00
11 (40000.0) 0.91 ± 0.00 0.90 ± 0.00 0.89 ± 0.00 0.88 ± 0.00
12 (60000.0) 0.94 ± 0.01 0.93 ± 0.00 0.92 ± 0.00 0.91 ± 0.00
13 (70000.0) 0.95 ± 0.01 0.94 ± 0.00 0.93 ± 0.00 0.91 ± 0.00

η, 4 (4.1)

p \ nTrk 1 (0.0) 2 (50.0) 3 (200.0) 4 (300.0)
1 (3000.0) nan 0.34 ± 1.65 0.24 ± 0.24 0.44 ± 0.28
2 (6000.0) nan -0.00 ± 0.00 0.68 ± 0.60 0.26 ± 0.23
3 (8000.0) nan 0.15 ± 0.19 -90.53 ± 15052.60 0.20 ± 0.18
4 (10000.0) nan 0.71 ± 1.10 0.00 ± 0.00 -0.25 ± 0.35
5 (12000.0) nan 0.01 ± 0.02 -0.00 ± 0.00 -0.00 ± 0.00
6 (14500.0) nan 0.28 ± 0.24 0.00 ± 0.00 -0.00 ± 0.00
7 (17500.0) nan -0.00 ± 0.00 -0.00 ± 0.00 -0.00 ± 0.00
8 (21500.0) 0.74 ± 0.06 0.71 ± 0.01 0.71 ± 0.02 0.71 ± 0.04
9 (27000.0) 0.79 ± 0.02 0.72 ± 0.00 0.69 ± 0.01 0.63 ± 0.01
10 (32000.0) 0.77 ± 0.01 0.74 ± 0.00 0.70 ± 0.00 0.65 ± 0.01
11 (40000.0) 0.76 ± 0.01 0.74 ± 0.00 0.71 ± 0.00 0.68 ± 0.00
12 (60000.0) 0.75 ± 0.01 0.71 ± 0.00 0.69 ± 0.00 0.67 ± 0.01
13 (70000.0) 0.69 ± 0.01 0.67 ± 0.00 0.66 ± 0.00 0.63 ± 0.00
yipengsun commented 2 years ago

For Mu_nopt sample:

[I 211016 00:00:41 make_eff_hists:179] Running PIDCalib2 make_eff_hists with the following config:
[I 211016 00:00:41 utils:116] ===============
[I 211016 00:00:41 utils:119] sample         : Turbo16
[I 211016 00:00:41 utils:119] magnet         : down
[I 211016 00:00:41 utils:119] particle       : Mu_nopt
[I 211016 00:00:41 utils:119] pid_cuts       : ['DLLmu>2.0&DLLe<1.0&IsMuon==1.0']
[I 211016 00:00:41 utils:119] bin_vars       : ['Brunel_P', 'Brunel_ETA', 'nTracks_Brunel']
[I 211016 00:00:41 utils:119] output_dir     : pidcalib_output
[I 211016 00:00:41 utils:119] verbose        : False
[I 211016 00:00:41 utils:119] version        : 1.0.1
[I 211016 00:00:41 utils:120] ===============
[I 211016 00:00:41 binning:185] No 'Brunel_P' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211016 00:00:41 binning:185] No 'Brunel_ETA' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211016 00:00:41 binning:185] No 'nTracks_Brunel' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211016 00:00:41 utils:450] Branches to be read: {'sWeight': 'probe_sWeight', 'DLLmu': 'probe_PIDmu', 'DLLe': 'probe_PIDe', 'IsMuon': 'probe_isMuon', 'Brunel_P': 'probe_Brunel_P', 'Brunel_ETA': 'probe_Brunel_ETA', 'nTracks_Brunel': 'nTracks_Brunel'}
[I 211016 00:00:41 utils:451] 28 calibration files from EOS will be processed
[I 211016 00:00:41 binning:185] No 'Brunel_P' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211016 00:00:41 binning:185] No 'Brunel_ETA' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211016 00:00:41 binning:185] No 'nTracks_Brunel' binning defined for particle 'Mu_nopt'. Falling back to particle 'Mu' binning.
[I 211016 00:05:17 utils:501] Processed 28/28 files                                                                   
[I 211016 00:05:17 utils:580] 56787766/60339195 (94.1%) events passed binning range cut
[I 211016 00:05:17 utils:580] 19814056/56787766 (34.9%) events passed 'DLLmu>2.0&DLLe<1.0&IsMuon==1.0' cut
[I 211016 00:05:17 make_eff_hists:215] Efficiency histograms saved to 'pidcalib_output/effhists-Turbo16-down-Mu_nopt-DLLmu>2.0&DLLe<1.0&IsMuon==1.0-Brunel_P.Brunel_ETA.nTracks_Brunel.pkl'
yipengsun commented 2 years ago

File: run2-rdx-2016-md-Mu_nopt-p_eta_ntracks_all_offline.root, Histo: eff Slicing axis: η

η, 1 (1.5)

p \ nTrk 1 (0.0) 2 (50.0) 3 (200.0) 4 (300.0)
1 (3000.0) 0.75 ± 0.00 0.74 ± 0.00 0.74 ± 0.00 0.73 ± 0.00
2 (6000.0) 0.78 ± 0.00 0.77 ± 0.00 0.77 ± 0.00 0.76 ± 0.00
3 (8000.0) 0.82 ± 0.01 0.81 ± 0.00 0.81 ± 0.00 0.80 ± 0.00
4 (10000.0) 0.84 ± 0.01 0.83 ± 0.00 0.83 ± 0.00 0.83 ± 0.00
5 (12000.0) 0.80 ± 0.01 0.80 ± 0.00 0.80 ± 0.00 0.81 ± 0.00
6 (14500.0) 0.81 ± 0.01 0.81 ± 0.00 0.82 ± 0.00 0.82 ± 0.00
7 (17500.0) 0.86 ± 0.01 0.88 ± 0.00 0.87 ± 0.00 0.88 ± 0.00
8 (21500.0) 0.92 ± 0.01 0.92 ± 0.00 0.92 ± 0.00 0.92 ± 0.00
9 (27000.0) 0.98 ± 0.01 0.95 ± 0.00 0.95 ± 0.00 0.95 ± 0.01
10 (32000.0) 0.95 ± 0.01 0.96 ± 0.00 0.96 ± 0.00 0.95 ± 0.01
11 (40000.0) 0.97 ± 0.02 0.96 ± 0.00 0.96 ± 0.00 0.97 ± 0.01
12 (60000.0) 0.94 ± 0.05 0.96 ± 0.01 0.95 ± 0.01 0.91 ± 0.01
13 (70000.0) 0.89 ± 0.04 0.94 ± 0.01 0.92 ± 0.01 0.90 ± 0.02

η, 2 (2.4)

p \ nTrk 1 (0.0) 2 (50.0) 3 (200.0) 4 (300.0)
1 (3000.0) 0.76 ± 0.01 0.75 ± 0.00 0.75 ± 0.00 0.73 ± 0.00
2 (6000.0) 0.93 ± 0.00 0.90 ± 0.00 0.88 ± 0.00 0.86 ± 0.00
3 (8000.0) 0.90 ± 0.00 0.88 ± 0.00 0.85 ± 0.00 0.83 ± 0.00
4 (10000.0) 0.87 ± 0.00 0.86 ± 0.00 0.85 ± 0.00 0.83 ± 0.00
5 (12000.0) 0.86 ± 0.00 0.85 ± 0.00 0.84 ± 0.00 0.82 ± 0.00
6 (14500.0) 0.87 ± 0.00 0.85 ± 0.00 0.84 ± 0.00 0.83 ± 0.00
7 (17500.0) 0.89 ± 0.00 0.88 ± 0.00 0.87 ± 0.00 0.87 ± 0.00
8 (21500.0) 0.93 ± 0.00 0.93 ± 0.00 0.92 ± 0.00 0.91 ± 0.00
9 (27000.0) 0.96 ± 0.00 0.95 ± 0.00 0.94 ± 0.00 0.93 ± 0.00
10 (32000.0) 0.97 ± 0.00 0.96 ± 0.00 0.96 ± 0.00 0.96 ± 0.00
11 (40000.0) 0.97 ± 0.00 0.97 ± 0.00 0.97 ± 0.00 0.96 ± 0.00
12 (60000.0) 0.97 ± 0.01 0.97 ± 0.00 0.96 ± 0.00 0.96 ± 0.00
13 (70000.0) 0.96 ± 0.01 0.96 ± 0.00 0.96 ± 0.00 0.95 ± 0.00

η, 3 (3.2)

p \ nTrk 1 (0.0) 2 (50.0) 3 (200.0) 4 (300.0)
1 (3000.0) 1.05 ± 0.05 0.96 ± 0.01 0.90 ± 0.03 0.96 ± 0.05
2 (6000.0) 0.97 ± 0.02 0.91 ± 0.01 0.88 ± 0.01 0.89 ± 0.02
3 (8000.0) 0.89 ± 0.01 0.85 ± 0.00 0.83 ± 0.01 0.79 ± 0.01
4 (10000.0) 0.84 ± 0.01 0.81 ± 0.00 0.79 ± 0.00 0.77 ± 0.01
5 (12000.0) 0.81 ± 0.01 0.79 ± 0.00 0.78 ± 0.00 0.75 ± 0.01
6 (14500.0) 0.79 ± 0.01 0.77 ± 0.00 0.74 ± 0.00 0.73 ± 0.00
7 (17500.0) 0.79 ± 0.00 0.77 ± 0.00 0.75 ± 0.00 0.72 ± 0.00
8 (21500.0) 0.82 ± 0.00 0.80 ± 0.00 0.78 ± 0.00 0.76 ± 0.00
9 (27000.0) 0.85 ± 0.00 0.83 ± 0.00 0.81 ± 0.00 0.79 ± 0.00
10 (32000.0) 0.87 ± 0.00 0.87 ± 0.00 0.85 ± 0.00 0.83 ± 0.00
11 (40000.0) 0.91 ± 0.00 0.90 ± 0.00 0.89 ± 0.00 0.88 ± 0.00
12 (60000.0) 0.94 ± 0.01 0.93 ± 0.00 0.92 ± 0.00 0.91 ± 0.00
13 (70000.0) 0.94 ± 0.01 0.94 ± 0.00 0.93 ± 0.00 0.91 ± 0.00

η, 4 (4.1)

p \ nTrk 1 (0.0) 2 (50.0) 3 (200.0) 4 (300.0)
1 (3000.0) 0.53 ± 0.18 0.84 ± 0.13 0.68 ± 0.22 0.54 ± 0.33
2 (6000.0) 1.21 ± 0.24 0.85 ± 0.05 0.86 ± 0.15 0.63 ± 0.18
3 (8000.0) 0.78 ± 0.07 0.93 ± 0.03 0.80 ± 0.06 0.80 ± 0.12
4 (10000.0) 0.93 ± 0.06 0.86 ± 0.02 0.83 ± 0.05 0.89 ± 0.11
5 (12000.0) 0.91 ± 0.05 0.85 ± 0.01 0.82 ± 0.03 0.90 ± 0.08
6 (14500.0) 0.82 ± 0.04 0.79 ± 0.01 0.76 ± 0.02 0.65 ± 0.03
7 (17500.0) 0.79 ± 0.03 0.73 ± 0.01 0.67 ± 0.01 0.63 ± 0.02
8 (21500.0) 0.75 ± 0.02 0.71 ± 0.00 0.67 ± 0.01 0.60 ± 0.01
9 (27000.0) 0.78 ± 0.02 0.70 ± 0.00 0.64 ± 0.01 0.59 ± 0.01
10 (32000.0) 0.77 ± 0.01 0.72 ± 0.00 0.67 ± 0.00 0.63 ± 0.01
11 (40000.0) 0.76 ± 0.01 0.73 ± 0.00 0.70 ± 0.00 0.67 ± 0.00
12 (60000.0) 0.75 ± 0.01 0.71 ± 0.00 0.69 ± 0.00 0.67 ± 0.01
13 (70000.0) 0.68 ± 0.01 0.67 ± 0.00 0.66 ± 0.00 0.63 ± 0.00
yipengsun commented 2 years ago

Hmmm, I think now even with just the Mu_nopt sample, the stats are more or less infinite (sans some high ETA, low P bins), so maybe we can just use the Mu_nopt sample exclusively and forget about the regular Mu sample?

manuelfs commented 2 years ago

Looks great, thank you!

This is a precision measurement, where we're trying to keep the total systematic uncertainty under 3%, so every bit counts. Do you know if there is a way to combine those samples? A weighted average may not be fully correct if there are correlated uncertainties.

Also, I see these samples are 2016 MD. Are we supposed to break down the weights by polarity?

yipengsun commented 2 years ago

I'll check the pidcalib2 instruction more to see if we can ask it to combine multiple samples to form a single weight file.

In terms of polarity, yes. The official samples are separated into year-polarity so we are supposed to do the same. Anyway I already have a script that can generate PID weights for all run 2 year-polarity so it's not an issue in terms of implementation.

manuelfs commented 2 years ago

I sent an email asking about this. I also found two discussions about the Mu_nopt sample that could be a bit useful: low PT Muon PIDCalib and Empty bins for Mu PIDCalib histogram

yipengsun commented 2 years ago

Hmm, the Mu and Mu_nopt samples have very different behaviors at eta_min = 3.2. For example, at low P, Mu samples are either nan or 0.0 +/- 0.0, whereas for Mu_nopt, the weights are very close to 1, with very reasonable uncertainties, so I don't think these samples can be combined naively.

yipengsun commented 2 years ago

After using the offline variables, the uncertainties at low eta are all very small for both Mu and Mu_nopt samples. For eta >= 3.2, the regular Mu is not very useful, whereas the Mu_nopt has better, but not ideal uncertainties, so combining the samples may not be very useful in terms of reducing uncertainties.

yipengsun commented 2 years ago

But polarity is a more interesting question. If we just combine Mu_nopt MagUp and MagDown, we certainly will have a smaller (~sqrt(2) better) uncertainty.

yipengsun commented 2 years ago

The PID weight has been integrated as into our MC workflow. For now, a demo workflow is provided, which uses FullSim 2016 MagDown, B -> D* Tau Nu.

To run the workflow:

git annex get ntuples/0.9.5-bugfix/Dst_D0-mc/Dst_D0--21_10_08--mc--MC_2016_Beam6500GeV-2016-MagDown-Nu1.6-25ns-Pythia8_Sim09j_Trig0x6139160F_Reco16_Turbo03a_Filtered_11574011_D0TAUNU.SAFESTRIPTRIG.DST__aux_hammer.root  # optional
git annex get ntuples/0.9.5-bugfix/Dst_D0-mc/Dst_D0--21_10_08--mc--MC_2016_Beam6500GeV-2016-MagDown-Nu1.6-25ns-Pythia8_Sim09j_Trig0x6139160F_Reco16_Turbo03a_Filtered_11574011_D0TAUNU.SAFESTRIPTRIG.DST.root

make rdx-ntuple-run2-mc-demo

The actual workflow is defined in:

https://github.com/umd-lhcb/lhcb-ntuples-gen/blob/8f26fc75bc9bb510f9605526968d3b48d714c4f7/workflows/rdx.py#L192-L197

yipengsun commented 2 years ago

For ProbNNk variable, we are supposed to use MC15TuneV1_ProbNN variables as indicated by PIDCalib wiki. This is also the default ProbNN variable in DaVinci and stripping, see this thread.

yipengsun commented 2 years ago

This is fully integrated in the RDX workflow.