Change Analysis Tools to Query DB for Calibration Constants

[bdorney]

Brief summary of issue

Superseding #154.

The following analysis scripts need calibration coefficients from the DB to analyze the data correctly:

• anaUltraScurve.py
• anaDACScan.py

The getVFAT3CalInfo(...) function provides a way to query the DB to retrieve the needed calibration constants needed for analysis.

Types of issue

• [ ] Bug report (report an issue with the code)
• [x] Feature request (request for change which adds functionality)

Expected Behavior

The DB can be queried using getVFAT3CalInfo(...) with an input list of VFAT chipID's from an input file. An example would be:

    from gempython.gemplotting.utils.dbutils import getVFAT3CalInfo
    dict_vfat3CalInfo[ohN] = getVFAT3CalInfo(dict_chipIDs[ohN],debug=args.debug)

Here the chipID's are stored in the input raw file, specifically the vfatID branch:

https://github.com/cms-gem-daq-project/vfatqc-python-scripts/blob/f80f955d03c50853f8931a578419ca4dd2f93f17/utils/treeStructure.py#L41-L42

Updating anaUltraScurve.py

The algorithm should:

1. Check if a local copy of the calibration file exists, if it does use it,
2. If a local copy does not exist, it should use the vfatID info in the input file to query the DB and retrieve the required calibration constants.

An example on how to implement the above is shown:

https://github.com/cms-gem-daq-project/vfatqc-python-scripts/blob/f80f955d03c50853f8931a578419ca4dd2f93f17/testConnectivity.py#L690-L706

Which for convenience is:

            # If the cal file exists parse it; otherwise write it from the DB query
            calFileCALDacName = "{0}/{1}/calFile_calDac_{1}.txt".format(dataPath,chamber_config[ohN])
            if os.path.isfile(calFileCALDacName):
                calDacInfo[ohN] = parseCalFile(calFileCALDacName)
            else:
                calFileCALDac = open(calFileCALDacName,"w")
                calFileADC.write("vfatN/I:slope/F:intercept/F\n")
                for idx,vfat3CalInfo in dict_vfat3CalInfo[ohN].iterrows():
                    calFileADC.write("{0}\t{1}\t{2}\n".format(
                        vfat3CalInfo['vfatN'],
                        vfat3CalInfo['cal_dacm'],
                        vfat3CalInfo['cal_dacb'])
                        )
                    pass
                calFileADC.close()
                calDacInfo[ohN] = (vfat3CalInfo['cal_dacm'],vfat3CalInfo['cal_dacb'])
                pass

Then one should also make isVFAT3 set to True by default and then set it to False only if you are analyzing VFAT2 data, to do this you should change the option:

https://github.com/cms-gem-daq-project/gem-plotting-tools/blob/78c88741fbe292f37476c79ffad108a137c1aa27/anaUltraScurve.py#L160-L161

To be:

    parser.add_option("--isVFAT2", action="store_true", dest="isVFAT2", default=False,
                      help="Provide this argument if input data was acquired from vfat2")

And then change all instances of the string options.isVFAT3 to just isVFAT3. Then add the line:

isVFAT3 = not options.isVFAT2

Right after line 187, i.e.

https://github.com/cms-gem-daq-project/gem-plotting-tools/blob/78c88741fbe292f37476c79ffad108a137c1aa27/anaUltraScurve.py#L187

Updating anaDACScan.py

The else block:

https://github.com/cms-gem-daq-project/gem-plotting-tools/blob/78c88741fbe292f37476c79ffad108a137c1aa27/utils/anautilities.py#L132-L138

Should be changed to query the DB and retrieve the necessary calibration coefficients. An example for how to do this is shown:

https://github.com/cms-gem-daq-project/vfatqc-python-scripts/blob/f80f955d03c50853f8931a578419ca4dd2f93f17/testConnectivity.py#L526-L539

Which is repeated here for convenience:

            # If the cal file exists do nothing; otherwise write it from the DB query
            calFileADCName = "{0}/{1}/calFile_{2}_{1}.txt".format(dataPath,chamber_config[ohN],adcName)
            if not os.path.isfile(calFileADCName):
                calFileADC = open(calFileADCName,"w")
                calFileADC.write("vfatN/I:slope/F:intercept/F\n")
                for idx,vfat3CalInfo in dict_vfat3CalInfo[ohN].iterrows():
                    calFileADC.write("{0}\t{1}\t{2}\n".format(
                        vfat3CalInfo['vfatN'],
                        vfat3CalInfo['{0}m'.format(adcName.lower())],
                        vfat3CalInfo['{0}b'.format(adcName.lower())])
                        )
                    pass
                calFileADC.close()
                pass

Current Behavior

Present anaUltraScurve.py

The calibration information comes from the input calibration file:

https://github.com/cms-gem-daq-project/gem-plotting-tools/blob/78c88741fbe292f37476c79ffad108a137c1aa27/anaUltraScurve.py#L201-L203

Present anaDACScan.py

The calibration information comes from the input calibration file list

https://github.com/cms-gem-daq-project/gem-plotting-tools/blob/78c88741fbe292f37476c79ffad108a137c1aa27/utils/anautilities.py#L121-L122

https://github.com/cms-gem-daq-project/gem-plotting-tools/blob/78c88741fbe292f37476c79ffad108a137c1aa27/utils/anautilities.py#L124-L132

Context (for feature requests)

The user needs to construct a calibration file for either CAL_DAC, ADC0, or ADC1 by hand and this is not ideal.

Your Environment

• Version used: 78c88741fbe292f37476c79ffad108a137c1aa27

[bdorney]

Partially addressed by #225 for the case of scurves.

DAC analysis still outstanding.

[lpetre-ulb]

Won't be fixed in legacy. testConnectivity.py and getCalInroFromDB.py can write the relevant files to the right location.