Improve calibration metadata in pdCIF files

[rowlesmr]

Originating from https://github.com/COMCIFS/Powder_Dictionary/pull/46#issuecomment-1367387642

I think once this (in conjunctionwith #46) is fleshed out, it will deprecate PD_CALIB, PD_CALIB_STD, and potentially PD_CALIB_OFFSET

Intensity

PD_CALIB_INTENSITY is a Loop category for intensity scaling. Its Loop as you need to be able to loop over all detectors. It's keyed on _pd_calib_intensity.id and _pd_calib_intensity.detector_id.

The category consists of:

• _pd_calib_intensity.scalar: (and *_su) the value by which _pd_meas.intensity_total values are multiplied by this to get _pd_proc.intensity_total. The source of this value could be from a calibration sample, or some beam intensity monitor, or something else - see special_details.
• _pd_calib_intensity.detector_id: To identify a detector in a multi-detector system - taken from _pd_calib.detector_id
• _pd_calib_intensity.detector_response: (and *_su) relative sensitivity of a detector in a multi-detector system - taken from _pd_calib.detector_response
• _pd_calib_intensity.id: To identify a particular intensity calibration.
• _pd_calib_intensity.phase_id: the phase_id of the phase used as the intensity std (if it was a sample).
• _pd_calib_intensity.phase_name: Identity of material used as a standard.
• _pd_calib_intensity.diffractogram_id: the diffractogram_id of the diffractogram collected of the the intensity std (if it was a sample).
• _pd_calib_intensity.block_id: the block_id of the data block containing the diffractogram and phase of the the intensity std (if it was a sample).
• _pd_calib_intensity.special_details: 'cause sometimes you need the space to give some info

Wavelength

PD_CALIB_WAVELENGTH is a Loop category for denoting the reference material from which the wavelength was determined (see also _diffrn_radiation_wavelength_determination). Its Loop as you may have multiple phases and diffractograms used in the calibration. It's keyed on _pd_calib_wavelength.id.

The category consists of:

• _pd_calib_wavelength.______: (and *_su) Is there are thing to go here? or do we just want to link to the calibrant?
• _pd_calib_wavelength.id: To identify a particular wavelength calibration.
• _pd_calib_wavelength.phase_id: the phase_id of the phase used as the wavelength std.
• _pd_calib_wavelength.phase_name: Identity of material used as a standard.
• _pd_calib_wavelength.diffractogram_id: the diffractogram_id of the diffractogram collected of the the wavelength std .
• _pd_calib_wavelength.block_id: the block_id of the data block containing the diffractogram and phase of the the wavelength std .
• _pd_calib_wavelength.special_details: 'cause sometimes you need the space to give some info

Angle / x-ordinate

But how do we deal with angle? There currently exists PD_CALIB_OFFSET, but this can only record a constant offset of 2Th; you can't record TOF, energy, position etc offsets, or offsets that vary with x-ordinate (eg specimen displacement)

PD_CALIB_ANGLE (is a horrible name, as it should also work for TOF, ED, ...) is a Loop category for for angle calibration. Its Loop as you need to be able to loop over all detectors. It's keyed on _pd_calib_angle.id and _pd_calib_angle.detector_id.

The category consists of:

• _pd_calib_angle.______: (and *_su) Is there are thing to go here? or do you just give _pd_meas.2theta and _pd_proc.2theta and be done with it?
• _pd_calib_angle.detector_id: To identify a detector in a multi-detector system - taken from _pd_calib.detector_id
• _pd_calib_angle.id: To identify a particular angle calibration.
• _pd_calib_angle.phase_id: the phase_id of the phase used as the angle std.
• _pd_calib_angle.phase_name: Identity of material used as a standard.
• _pd_calib_angle.diffractogram_id: the diffractogram_id of the diffractogram collected of the the angle std .
• _pd_calib_angle.block_id: the block_id of the data block containing the diffractogram and phase of the the angle std .
• _pd_calib_angle.special_details: 'cause sometimes you need the space to give some info
• more data names relating to offsets and other variations in the various X-coordinates, ...

[rowlesmr]

being addressed in #55

[rowlesmr]

OK, rereading PD_CALIB_OFFSET, and we can deal with non-constant offsets (eg specimen displacement) by enumerating a constand offset for every data point.

X-coordinate

But how do we deal with the x-coordinate? There currently exists PD_CALIB_OFFSET, but this can only record a constant offset of 2Th (need to loop for non-constant offsets), and you can't record TOF or position offsets.

PD_CALIB_XCOORD is a Loop category for for x-coordinate calibration. Its Loop as you need to be able to loop over all detectors, as well as looping to provide an offset to every data point. It's keyed on _pd_calib_angle.id and _pd_calib_angle.detector_id. Does _pd_calib_xcoord.point_id also need to be in there? It is able to record calibrations for 2theta, position, and TOF.

The category consists of these basic data items:

• _pd_calib_xcoord.block_id: the block_id of the data block containing the diffractogram or phase used for x-coord calibration, if a reference material was used.
• _pd_calib_xcoord.detector_id: To identify a detector in a multi-detector system
• _pd_calib_xcoord.diffractogram_id: the diffractogram_id of a diffractogram used for x-coord calibration, if a reference material was used. It is possible that more than one diffractogram was used
• _pd_calib_xcoord.id: To identify a particular angle calibration.
• _pd_calib_xcoord.phase_id: the phase_id of a phase used for x-coord calibration, if a reference material was used. This phase can be an internal or external reference material. It is possible that more than one phase was used.
• _pd_calib_xcoord.phase_name: Identity of material(s) used as a reference material, if a reference material was used. .
• _pd_calib_xcoord.special_details: 'cause sometimes you need the space to give some info

and these specific data items:

• _pd_calib_xcoord.point_id: to identify the specific point to which an offset applied.
• _pd_calib_xcoord.2theta: offset angle used to calibrate angle as defined in _pdmeas.2theta*. The offset is added.
• _pd_calib_xcoord.2theta_nominal: the nominal angle to which the associated offset applies.
• _pd_calib_xcoord.position: offset position used to calibrate position as defined in _pd_meas.position. The offset is added.
• _pd_calib_xcoord.position_nominal: the nominal position to which the associated offset applies.
• _pd_calib_xcoord.time_of_flight: offset time_of_flight used to calibrate time_of_flight as defined in _pd_meas.time_of_flight. The offset is added.
• _pd_calib_xcoord.time_of_flight_nominal: the nominal time_of_flight to which the associated offset applies.

I haven't included the equivalent of _pd_calib.2theta_off_min/max in here. Should they be replicated for each x-coordinate?

Alternatively, have a different PD_CALIB_* for each 2theta, position, TOF, and channel num.

[rowlesmr]

Sort of related to angle calibrations is how to record specimen displacements.

0. Do we need to?
1. I think there are three displacements (units of mm): edit - reading back, even these definitions are inadequate...
   • Equatorial - Displacement perpendicular to the beam, in the plane of diffraction; ie specimen displacement in Bragg-Brentano
   • Lateral - Displacement parallel to the beam; ie increasing/decreasing the effective specimen-detector distance. You can get this with capillaries in Debye-Scherrer
   • Axial - Displacement perpendicular to the beam, and perpendicular to the plane of diffraction. (if equatorial is up/down, and lateral is left/right, axial is toward/away from you)
2. These displacements also need to have specimen-detector distances recorded. I think that appropriate data items would be
   • _pd_instr.detector_circle_radius
   • _pd_instr.dist_spec_anal
   • _pd_instr.dist_spec_detc
   • _pd_instr.dist_spec_vdetc
3. Where to put such displacements? I think PD_SPEC is the best place. I don't think the PD_INSTR is appropriate. PD_PROC_LS doesn't seem right, as although it is often a refined parameter, it does refer to a "real" thing. PD_SPEC is a Set category; do we need to be able to provide displacements per detector_id? if so, then we'd need to make something like PD_SPEC_DISPLACEMENT or PD_CALIB_DISPLACEMENT .
4. Can we provide something like a _pd_spec.displacement_geometry to allow for the automated calculation of corrected 2theta values ? With an enumerated choice: "bragg-brentano", "debye-scherrer", "seeman-bohlin", ... , "other". Other would require _pd_calibration.conversion_eqn to be defined.

(I promise to stop thinking of things now... :))

[rowlesmr]

see also #91 #92 #93