cms-mual / MuAlPhysicsValidation

Muon Alignment Physics Validation
0 stars 4 forks source link

Performance plot histogrammar plots deviate from the vanilla pyroot code #11

Open rymuelle opened 6 years ago

rymuelle commented 6 years ago

As Adrian has noted, we see some discrepancies between the two approaches in some plots.

e.g.

image

vs.

image

One cause was simply due to using the absolute value of eta in the histogrammar implementation. This is fixed. Some other plots have more subtle issues.

Its worth noting that some plots are exact:

e.g.

iterm2 lg3tim th1f_sta_trk_delta_phi_endcap

Some are similar, but not exact:

iterm2 lztf9z glb_sta_pt_v_ptres_gaus_sigma

And some are wildly different:

iterm2 69qbtq sta_glb_pt_hybridsta_mass_gaus_mean

rymuelle commented 6 years ago

Ok, so I found and fixed one issue with the mass plots. The selection criteria used the hybrid mass rather than the global mass causing an issue.

Now the plots look quite similar:

iterm2 9ggjym sta_glb_eta_hybridsta_mass_gaus_mean

iterm2 nx4gia sta_glb_eta_hybridsta_mass_gaus_sigma

But it isn't clear if they are the same. One issue is that since both have a random element there is some noise. I would be ok with accepting some noise, except the vs pt graph looks terrible:

iterm2 cq3kdz sta_glb_pt_hybridsta_mass_gaus_sigma

demarley commented 6 years ago

@rymuelle has fixed many of these issues, I'm currently testing it.

demarley commented 6 years ago

From what I can tell, the results are consistent between the two frameworks.
Suggestions:

  1. Different marker styles for each type of plot so that you can see them even if they are the same value
  2. File naming is inconsistent. For example, the pulls plots are named TH2F_glb_sta_eta_ptPull_gaus_mean.png but the res plots are named glb_sta_eta_v_ptRes_gaus_mean.png.
  3. I generally discourage the use of code like this because it obfuscates where functions / variables exist and makes it difficult to repurpose code (IMO the code should be object-oriented -- put in classes -- instead).