updates to calibrateThrDac.py fits: removal of low thrDac values sensitive to pedestal and use of non-linear function

[AndrewLevin]

These changes address poor quality scurve mean vs thrdac fits due to non-linearity and pedestal effects.

Description

I have changed the scurve mean fitting function to be a cubic polynomial and removed the 1 or 2 lowest thrDac values based on whether the width is consistent within one error bar with the mean of the widths of the largest 3 thrDac values.

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [x] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)

Motivation and Context

This was requested in https://github.com/cms-gem-daq-project/gem-plotting-tools/issues/184

How Has This Been Tested?

Yes, I tested it on ARM DAC calibration data for GE11-X-L-GHENT-0002. Scurve mean fits:

Scurve sigma fits:

Screenshots (if appropriate):

Checklist:

• [x] My code follows the code style of this project.
• [ ] My change requires a change to the documentation.
• [ ] I have updated the documentation accordingly.
• [ ] I have added tests to cover my changes.
• [ ] All new and existing tests passed.

[bdorney]

This isn't really going to solve the issue mentioned here.

Take the case of VFAT18. Here you're excluding the the ARM DAC calibration data below 6 fC. However the chip is going to be used explicitly below this level because the noise is "low enough." See sbit rate measurement on slide 17.

Then when VFAT18 is trimmed at this ARM DAC setting the trim algorithm fails to align the channels because the fit function deviates from the true value at this point.

And this is exactly what will happen again. So I think we need to revisit how you're removing this point.

For example if I look at the ENC measurements ARM DAC of 40 is still well controlled. And ideally there would have been a measurement point at ARM DAC of 35 but there isn't. But while the ENC @ 30 is large the spread the means are still controlled. So this point should still be good.

Similar comments apply to VFATs 1, 4, 6, 7 and 14.

VFAT10 has a point that the algorithm should try to "ignore" if possible (separate PR could be made for this though).

I would even use VFAT8.

The ENC fit could use a separate fit range from the mean (and should probably). But out of this data if I was fitting this by hand the only one I would not use for the ARM DAC Calibration would be the first point of VFAT14.

Also I would not use an odd polynomial for exactly what happens in VFAT15. That curved behavior at low ARM DAC is not real and trimming from that will not be successful.

TL;DR, our initial idea of how to choose the fit range was not successful and a second attempt/more advanced algorithm needs to be made.

You could for example if the ENC is larger than the asymptotic behavior then check if the corresponding scurve mean is an outlier (see functions in utils/anautilities.py) and try to remove this point. Or you could try a finite difference method to see what an expected point is vs. the real. Further development could be needed.

Additionally you might have to also improve the fitting of the scurve mean distributions themselves. And also the ENC distributions. For example the ENC is fit with a Gaussian but it is not normally distributed (e.g. it can never be 0). And in many cases the ENC is not symmetric and a Gaussian fit converges on a local minima rather than a global one and this causes the point to be reported to not work. Here something like a Chi^2 distribution could be used as a fit, or trying the mean, median, or MPV of the dataset. Somethings to consider.

[AndrewLevin]

I just committed two changes:

1) I stitched together two linear functions with a smooth transition region between thrDac = 41 and 49 2) I only remove low thrDac values if the scurve mean is > 30

Here are the updates scurve mean vs thrDac fits for GE11-X-L-GHENT-0002:

[bdorney]

I just committed two changes:

1. I stitched together two linear functions with a smooth transition region between thrDac = 41 and 49

2. I only remove low thrDac values if the scurve mean is > 30

Here are the updates scurve mean vs thrDac fits for GE11-X-L-GHENT-0002:

Please ensure the fit is a monotonically increasing function and is continuous. Again those jumps in the fit curve are going to cause problems during the trim algorithm. Why not just try an even polynomial?

I only remove low thrDac values if the scurve mean is > 30

And what happens if the scurve mean really is 30? There have been several cases in which VFATs with high charge to comp value correspondence exist, see for example the following cases:

Specifically the following short detectors:

• Bari 0006, VFAT14,
• CERN 0010, VFAT5 & 10
• INDIA 0007, VFAT14,
• CERN 0009, VFAT7 & 11

The goal should not be to try to deliver on this request as fast as possible with minimum effort but to try to design an algorithm that will not fail and handle 99.99....99% of call cases (like the scurve fit routine).

Try to study these and come up with a better approach. Then you can reserve a setup to see how your improvements to the algorithm impact the trim results.

[AndrewLevin]

Here is what I get with a quartic polynomial:

Here is what I get with a quadratic polynomial:

For VFAT 11 the fit looks bad because the shape is more like two linear functions stitched together than a polynomial and for VFAT15 the fit is highly overfit in the quartic case

I am only considering the first and second thrDac values for removal if both the scurve mean is > 30 and the scurve width is not consistent with the average of the scurve width of the high thrDac. In your examples above, the scurve mean > 30 only at high thrDac values, as far as I noticed.

[bdorney]

For VFAT 11 the fit looks bad because the shape is more like two linear functions stitched together than a polynomial and for VFAT15 the fit is highly overfit in the quartic case

What happens if you add an scurve point at ARM DAC = 35? e.g. you'll see the knee like in the other positions.

I am only considering the first and second thrDac values for removal if both the scurve mean is > 30 and the scurve width is not consistent with the average of the scurve width of the high thrDac.

Additionally what is "the first and second" ARM DAC here? Is it the first and second in the dataset (which could change depending on the scan) or some ARM DAC below X? What is high ARM DAC considered as here?

In your examples above, the scurve mean > 30 only at high thrDac values, as far as I noticed.

My point was your choice seemed somewhat arbitrary, and that a VFAT could (out of the 24 * 161 = 3864 VFATs in GE1/1, not considering GE2/1 or ME0) have some interesting fringe cases that the algorithm should treat, and correctly; I'm not confident that this approach will succeed in all cases yet. You're not going to go back and fit N failed VFATs, or poorly fit ones, by hand in p5.

[AndrewLevin]

What happens if you add an scurve point at ARM DAC = 35? e.g. you'll see the knee like in the other positions.

Yes, which function to fit with depends on the number of thrDac values. What assumptions can I make about the number and locations of the thrDac values? Should I take a new scurve scan at ARM DAC = 35 for GE11-X-L-GHENT-0002?

Additionally what is "the first and second" ARM DAC here? Is it the first and second in the dataset (which could change depending on the scan) or some ARM DAC below X? What is high ARM DAC considered as here?

I was assuming that they were sorted. I just made a commit that sorts them explicitly. I defined high ARM DAC as the highest three ARM DAC values.

EDIT: I just made another commit which requires the thrDac value to be < 50 in order to be considered for removal. So, I am now removing thrDac values that satisfy all of: 1) thrDac < 50, 2) scurve mean > 30, 3) scurve width not consistent with the mean of the scurve widths for the three highest thrDac values, 4) thrDac not one of the three highest thrDac values

My point was your choice seemed somewhat arbitrary, and that a VFAT could (out of the 24 * 161 = 3864 VFATs in GE1/1, not considering GE2/1 or ME0) have some interesting fringe cases that the algorithm should treat, and correctly; I'm not confident that this approach will succeed in all cases yet. You're not going to go back and fit N failed VFATs, or poorly fit ones, by hand in p5.

Yes, 30 is pretty arbitrary. Should I make it less arbitrary by calculating the mean and standard deviation for the VFATs we are testing with and set it 5 standard deviations above the mean?

[bdorney]

I think the best thing to do would be to try to:

1. retake an arm dac calibration run using the GE1/1 on the coffin setup (you can test https://github.com/cms-gem-daq-project/sw_utils/pull/19 then),
2. analyze the arm dac calibration, and
3. use the new calibration to trim the detector

When the hybrid production is finalized you can also use GE1/1-X-L-GHENT-0002 but since this interferes with the eff measurements it will have to wait until next week.

[bdorney]

Another interesting case to consider would be:

/data/bigdisk/GEM-Data-Taking/GE11_QC8/GE11-X-L-CERN-0002/armDacCal/2019.02.22.02.43

[bdorney]

@AndrewLevin any updates here? Would you be able to test this all out on the coffin setup?

[AndrewLevin]

I can start testing this afternoon. I just requested a reservation.

[bdorney]

Any updates here? At minimum this will need a rebase.

[AndrewLevin]

For the ARM DAC calibration scans I took with the coffin detector, a linear fit is perfect.

So, in order to further develop and validate the non-linear fitting, I guess I would need to take an ARM DAC calibration scan on GE1/1-X-L-GHENT-0002?

[bdorney]

You don’t need to take scans per say; you can use old data. We have multiple detectors worth of old data all under DATA_PATH. You could make a survey of this old data and see how it works.

[AndrewLevin]

OK, I will first try find existing data that shows the non-linearity that I can fit. Is GE1/1-X-L-GHENT-0002 actually on one of the setups or it would need to be setup?

[bdorney]

OK, I will first try find existing data that shows the non-linearity that I can fit. Is GE1/1-X-L-GHENT-0002 actually on one of the setups or it would need to be setup?

The detector is no longer setup and the original set of VFATs used by this detector would have to be tracked down. So please first go through the data on $DATA_PATH there are at least 5 or 10 detectors worth of data from the past year so there shouldn't be a need to retake old data.

[AndrewLevin]

I did not find any ARM DAC calibration data in $DATA_PATH that has the low ARM DAC behavior of GE11-X-L-GHENT-0002 VFATs 11 and 12. Many have the opposite low ARM DAC behavior, concave up instead of concave down.

I fit all of the ARM DAC calibration data in $DATA_PATH with a quartic polynomial and with a quadratic polynomial:

https://amlevin.web.cern.ch/amlevin/calibrateThrDacQuarticPlots/ https://amlevin.web.cern.ch/amlevin/calibrateThrDacQuadraticPlots/

The quadratic polynomial does not model well the concave up part at low ARM DAC (see, for example, VFAT 11 in https://amlevin.web.cern.ch/amlevin/calibrateThrDacQuadraticPlots/canvScurveMeanVsThrDac_Summary.28.png), so I would suggest that we go with the quartic polynomial.

[AndrewLevin]

I just updated the algorithm to remove points from the scurve mean fit. Points are now removed if they satisfy any of the following conditions:

1) scurveMean < 0.1 2) scurveMean > 6 and thrDac < 50 and scurvemean > lastUnremovedScurvemean (the points are considered for removal from high thrDac value to low thrDac value) and scurveSigma not consistent with 5 highest scurveSigma values 3) scurveMean > 6 and thrDac < 50 and scurveMean > 2*lastUnremovedScurveMean 4) scurveMean > 45 and thrDac < 60

Updated fits are here: https://amlevin.web.cern.ch/amlevin/calibrateThrDacQuarticPlots/

[bdorney]

If this is considered final could you make a summary for tomorrow's SW dev meeting that describes:

• the changes to the algorithm made,
• how points are removed from analysis,
• example results;
  • Take a linear case and a nonlinear case,
  • Include data/fit distributions, and
• Data/fit distributions from all analyzed plots as a single histogram showing the stats box

Target length should be 10 minutes.

Following that you could cleanup the git history by squashing as there are several unnecessary commits and then we could approve this PR.

[AndrewLevin]

Yes, I will prepare some slides for the SW developers meeting tomorrow.

[jsturdy]

Following that you could cleanup the git history by squashing as there are several unnecessary commits and then we could approve this PR.

A quick look at the graph indicates that there isn't really any reason to not squash this completely

[AndrewLevin]

done

[AndrewLevin]

I updated the fit function such that it is expanded about the lower edge of the fit range (by default 0), and required to be increasing at this point. This seems to resolve the non-monotonicity issues discussed in the meeting yesterday. For example, see VFAT15 in Example 33 old, new and VFAT21 in Example 4 old, new.

[bdorney]

I updated the fit function such that it is expanded about the lower edge of the fit range (by default 0), and required to be increasing at this point. This seems to resolve the non-monotonicity issues discussed in the meeting yesterday. For example, see VFAT15 in Example 33 old, new and VFAT21 in Example 4 old, new.

It seems the link for example 4 is the same as 33. Could you update the links?

[AndrewLevin]

done