search

cms-gem-daq-project / gem-plotting-tools

Repository for GEM commissioning plotting tools
GNU General Public License v3.0
1 stars 26 forks source link

DAC scan analysis fails to produce good plots #262

Closed lpetre-ulb closed 4 years ago

lpetre-ulb commented 4 years ago

Brief summary of issue

While testing @dteague's PR for GE2/1 generalizing at the FIT setup, we (@sbutalla and I) noticed unexpected DAC scans plots.

Nevertheless, the VFATs seem to be properly biased since the nominal value fo all DACs can be reached and the Scurves look "normal".

Types of issue

Expected Behavior

As an example, the BIAS_SD_I_BFCAS plots for GE1/1 before the PR look like this:

SummaryTestStand0_DACScan_CFG_BIAS_SD_I_BFCAS

Current Behavior

However, with the PR and for GE2/1, the plots look like this:

SummaryGE21-M5-P1_DACScan_CFG_BIAS_SD_I_BFCAS

The title, axis names and axis ranges are incorrect.

Steps to Reproduce (for bugs)

  1. Run testConnectivity.py, including the DAC scans, with a GE2/1 chamber (possibly also affects GE1/1).
  2. Look at the resulting DAC scan plots.

Your Environment

lpetre-ulb commented 4 years ago

Confirmed for GE1/1 and the latest commit in develop:

SummaryTestStand10_DACScan_CFG_BIAS_SD_I_BFCAS

Moreover, the DAC scan analysis during QC@ULB also fails with many "large chiquare" errors while the fits used to be very good. All VFATs are affected:

TTree::SetCacheSizeAux:0: RuntimeWarning: A TTreeCache could not be created because the TTree has no file
Initializing TObjects
Looping over stored events in dacScanTree
fitting DAC vs. ADC distributions
Warning: large chisquare for VFAT0 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT1 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT2 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT3 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT4 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT5 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT6 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT7 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT8 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT9 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT10 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT11 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT13 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT14 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT15 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT16 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT17 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT18 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT19 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT20 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT21 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT22 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT23 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_PRE_VREF.
Warning: large chisquare for VFAT0 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_SD_I_BDIFF.
Warning: large chisquare for VFAT1 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_SD_I_BDIFF.
Warning: large chisquare for VFAT2 of chamber TestStand10 (Shelf1,Slot4,OH10) DAC CFG_BIAS_SD_I_BDIFF.
[...]

Investigation is required; possibly related to issues seen during commissioning at P5.

jsturdy commented 4 years ago

Can you provide the input files for these so that the people (@AndrewLevin, @bregnery, @dteague) who put recent changes in can do a bisect to find the offending commit?

lpetre-ulb commented 4 years ago

Here is an archive containing both the raw and analyzed root files as well as the chipIDs and calibration constants for ADC0 (non Reed-Muller encoded chipID): BadDACScanPlots.zip

Looking at the analyzed data, the TGraphErrors plots are good for each VFAT, but the TF1 ones are not.

Also, the fits are rather good. The failing fits are those which become significantly nonlinear for high DAC values. The chi²/NDF seems not too bad though; it should be probably used instead of the chi² since the NDF changes between the scans.

AndrewLevin commented 4 years ago

The data in that zip file is for TestStand6, not TestStand10, and I do not find any large chi squares other than for CFG_THR_ZCC_DAC (which is because this register has both a voltage and current mode, and both are incorrectly fit on the same plot (https://github.com/cms-gem-daq-project/gem-plotting-tools/blob/ca99453e7e54bf935eab5e3e94f786a8b9657153/utils/anaInfo.py#L31)).

The code to create the summary plot was rewritten in the GE11 generalizing pull request, and it now is incorrectly only plotting the fitted function, not the data. @dteague, can you try to fix it?

lpetre-ulb commented 4 years ago

The data in that zip file is for TestStand6, not TestStand10,

Right, I've taken a new set of data with the TestStand6 which was available at the moment. The DAC errors are similar though.

and I do not find any large chi squares

Indeed, they are not particularly large, but some are larger than the default of 100. For instance, BIAS_SH_I_BDIFF for the VFAT0 (chi²/NDF = 2429/250).

https://github.com/cms-gem-daq-project/gem-plotting-tools/blob/ca99453e7e54bf935eab5e3e94f786a8b9657153/utils/anaInfo.py#L15-L16

Since the number of points is is not equal for all DACs, my proposal was to use chi²/NDF instead of chi².

other than for CFG_THR_ZCC_DAC (which is because this register has both a voltage and current mode, and both are incorrectly fit on the same plot

Correct, but these (THR_ZCC, but also THR_ARM) are not scanned by testConnectivity.py anyway, only by run_scans.py dacScanV3. So, they are not an issue. The failure is even expected and shows that the check fulfills its purpose.

The code to create the summary plot was rewritten in the GE11 generalizing pull request, and it now is incorrectly only plotting the fitted function, not the data. @dteague, can you try to fix it?

:+1:

AndrewLevin commented 4 years ago

The data in that zip file is for TestStand6, not TestStand10,

Right, I've taken a new set of data with the TestStand6 which was available at the moment. The DAC errors are similar though.

and I do not find any large chi squares

Indeed, they are not particularly large, but some are larger than the default of 100. For instance, BIAS_SH_I_BDIFF for the VFAT0 (chi²/NDF = 2429/250).

https://github.com/cms-gem-daq-project/gem-plotting-tools/blob/ca99453e7e54bf935eab5e3e94f786a8b9657153/utils/anaInfo.py#L15-L16

Since the number of points is is not equal for all DACs, my proposal was to use chi²/NDF instead of chi².

Yes, sorry, I see the large chi square values now. I agree we should use chi²/NDF instead of chi². I guess we can make the cutoff somewhere between 15-25.

dteague commented 4 years ago

Sorry, I was flying back to the US, so that's why I've been MIA. I'll look into this today and see if I can fix this

jsturdy commented 4 years ago

@dteague do you have any update on this yet?