Add RE metadata to image

[prjemian]

Add the RE.md dictionary items to the area detector image attributes.

[prjemian]

One way that adds them to the HDF5 file is to write them into the layout.xml file. Here's an example file in the BlueskyMetadata:NXcollection group:

<?xml version="1.0" standalone="no" ?>
<hdf5_layout>
  <group name="entry">
    <attribute name="NX_class" source="constant" value="NXentry" type="string"/>
    <group name="instrument">
      <attribute name="NX_class" source="constant" value="NXinstrument" type="string"/>
      <group name="detector">
        <attribute name="NX_class" source="constant" value="NXdetector" type="string"/>
        <dataset name="data" source="detector" det_default="true">
          <attribute name="NX_class" source="constant" value="SDS" type="string"/>
          <attribute name="signal" source="constant" value="1" type="int"/>
          <attribute name="target" source="constant" value="/entry/instrument/detector/data" type="string"/>
        </dataset>
        <group name="NDAttributes">
          <dataset name="ColorMode" source="ndattribute" ndattribute="ColorMode"/>
        </group>          <!-- end group NDAttribute -->
      </group>            <!-- end group detector -->
      <group name="NDAttributes" ndattr_default="true">
        <attribute name="NX_class" source="constant" value="NXcollection" type="string"/>
      </group>            <!-- end group NDAttribute (default) -->
      <group name="BlueskyMetadata">
        <attribute name="NX_class" source="constant" value="NXcollection" type="string"/>
        <attribute name="string" value="example text" source="constant" type="string"/>
        <attribute name="int" value="1" source="constant" type="int"/>
        <attribute name="float" value="1.2345" source="constant" type="float"/>
        <attribute name="double" value="1.2345" source="constant" type="double"/>
      </group>            <!-- end group NDAttribute (default) -->
      <group name="performance">
        <dataset name="timestamp" source="ndattribute"/>
      </group>            <!-- end group performance -->
    </group>              <!-- end group instrument -->
    <group name="data">
      <attribute name="NX_class" source="constant" value="NXdata" type="string"/>
      <hardlink name="data" target="/entry/instrument/detector/data"/>
      <!-- The "target" attribute in /entry/instrument/detector/data is used to
           tell Nexus utilities that this is a hardlink -->
    </group>              <!-- end group data -->
  </group>                <!-- end group entry -->
</hdf5_layout>

and it's tree representation:

  entry:NXentry
    @NX_class = "NXentry"
    data:NXdata
      @NX_class = "NXdata"
      data --> b'/entry/instrument/detector/data'
    instrument:NXinstrument
      @NX_class = "NXinstrument"
      BlueskyMetadata:NXcollection
        @NX_class = "NXcollection"
        @double = "1.2345"
        @float = 1.2345
        @int = 1
        @string = "example text"
      NDAttributes:NXcollection
        @NX_class = "NXcollection"
...

[prjemian]

note: attempts to write constant double or integer values to <dataset elements were always written as string.

[prjemian]

It's trickier to write them as constants in the NDAttributes since that requires either EPICS PVs or a custom c++ function. Needs a some research.

[prjemian]

Also note that only simply data types (int, float, str, bool) should be written. A dict, list, or tuple will need to be broken down into its simpler members. Some items in RE.md might not be expressed.

-Or-

Write All of the RE.md as JSON or YAML into a single long string PV which is then easier to describe in the attributes.xml file. Content must be parsed when file is read, so this is not optimal.

[prjemian]

General documentation: https://areadetector.github.io/master/ADCore/NDArray.html#pvattribute, types can be from this list:

• EPICS_PV
• FUNCT (ADCore contains example code, myAttributeFunctions.cpp, that demonstrates how to write such functions. )
• PARAM

Summary

To use a FUNCT type of NDAttribute, custom c++ code is required to built into the AD instance. Since Bluesky cannot control the IOC (in the general case), this is not a possibility for bluesky. Note also the NDattributes.xsd (XML Schema) does not allow type="FUNCT" as valid. Probably an oversight there.

[prjemian]

Might be nice to extend ADCore to allow constant NDAttributes to be defined directly in an attributes.xml file.

XML requires its content to be defined in strings but the datatype could be used to cast that content to the desired type, failing the load of the file if this step fails.

With this enhancement, a client such as bluesky could inject its metadata into every image without need for a custom function in the IOC.

[prjemian]

As an alternative, a set of PVs would be constructed prepared for this purpose. Not as flexible. Type (RTYP) of each PV would need to match.

[keenanlang]

As far as I can tell, all the infrastructure to support this is already all built into ADCore, it's just limitations on the attributes file parser that is holding things back. There's a set of if-then-elseif statement that compares the value of the "source" attribute with a list of strings to see what type of NDAttribute to create. However, a basic NDAttribute would be able to hold a regular constant value and would just need an entry in the if-then-elseif chain to enable it.

Replacing lines 461-463 of asynNDArrayDriver.cpp with:

this->pAttributeList->add(pName, pDescription, NDAttrString, pSource);

Should be all that's necessary to allow a line like this in the xml:

\<Attribute name="A Constant" type="CONST" source="A Value" />

Though, it would be a decent amount more work to do all the string to value parsing if there was a need to have more than just strings.