REQ Improve handling of enums and bitmaps

ergo-furrer commented 1 year ago

User Voice

Drawing from our experience, with heat pumps as an illustrative example, we have encountered substantial variations in enum and bitmap definitions across different products. Frequently, their meanings resist standardization for inclusion in a generalized functional profile.

The present XSD schema delineates all established product-specific enum and bitmap mappings. However, it is worth considering that, from a conceptual perspective, particulars exclusive to each product should find representation within the EI-XML. It is advisable to refrain from embedding product-specific information directly into the XSD. Such an approach would necessitate recurrent schema updates for every novel product accommodating fresh enum or bitmap values.

Hence, the XSD should be designed to facilitate a versatile enum and bitmap mapping mechanism within the EI-XML, while completely excluding any product-specific elements from the XSD itself.

Suggested Solution

Enum's

The XSD defines an enum such that the EI-XML for the generic data points reads as follows:

<sgr:dataPoint datapointName="CurrentDirL1" mroVisibilityIndicator="O" rwpDatadirection="R" unit="NONE">
    <sgr:dataType>
          <sgr:enum>
            <sgr:enumMapping>         
              <sgr:literal>DWH_OFF</sgr:literal><sgr:ordinal>0</sgr:ordinal>
              <sgr:literal>DWH_ON</sgr:literal><sgr:ordinal>1</sgr:ordinal>
              <sgr:literal>DWH_EMERGENCY</sgr:literal><sgr:ordinal>2</sgr:ordinal>
              <sgr:hexMask>03</sgr:hexMask>
            </sgr:enumMapping>
          </sgr:enum>
      </sgr:dataType>
</sgr:dataPoint>
<sgr:modbusDataPoint>
      <sgr:modbusDataType><sgr:int8U>0</sgr:int8U></sgr:modbusDataType>
      <sgr:modbusFirstRegisterReference registerType="HoldRegister" addr="16402" bitRank="0" />
      <sgr:dpSizeNrRegisters>1</sgr:dpSizeNrRegisters>
      <sgr:masterFunctionsSupported>Primitives</sgr:masterFunctionsSupported>
</sgr:modbusDataPoint>

<sgr:ordinal> is numeric 0, 1, 2 ......
each enum literal (string) value is mapped to an integer value and vice versa.

The according XSD is as follows:

 <complexType name="enumType">
    <sequence>
      <element name="enumMapping" minOccurs="0" maxOccurs="unbounded" type="sgr:enumMappingType" />
      <element name="hexMask" minOccurs="0" maxOccurs="1" type="sgr:hexType" />
      <element name="enumVal" minOccurs="0" maxOccurs="1" type="sgr:enumMappingType" />
    </sequence>
  </complexType>

  <complexType name="enumMappingType">
    <sequence>
      <element name="literal" minOccurs="1" maxOccurs="1" type="string" />
      <element name="ordinal" minOccurs="1" maxOccurs="1" type="int" />
    </sequence>
  </complexType>

The XSD contains an additional element enumVal. enumVal used by the SGrJava implementation to represent the read/written enumeration value, since the generic API returns the XSD generated classes (and therefore needs a container for the read/write value).
Returning the effective value and the mapping-table within the Java API has the advantage that the possible enum values are documented through the API...

Bitmap's

The XSD defines a bitmap such that the EI-XML for generic data points reads as follows:

   <sgr:dpListElement>
     <sgr:dataPoint>
       <sgr:dataType>
         <sgr:bitmap>
           <sgr:mapping><sgr:literal>STATUS_INDICATOR_A</sgr:literal><sgr:hexMask>0001</sgr:hexMask></sgr:mapping>
           <sgr:mapping><sgr:literal>STATUS_INDICATOR_B</sgr:literal><sgr:hexMask>0080</sgr:hexMask></sgr:mapping>
           <sgr:mapping><sgr:literal>STATUS_INDICATOR_C</sgr:literal><sgr:hexMask>0400</sgr:hexMask></sgr:mapping>
         </sgr:bitmap>
       </sgr:dataType>
     </sgr:dataPoint>
     <sgr:modbusDataPoint>
       <sgr:modbusDataType>
         <sgr:int16U>0</sgr:int16U>
       </sgr:modbusDataType>
       <sgr:modbusFirstRegisterReference addr="16406" registerType="HoldRegister"/>
       <sgr:dpSizeNrRegisters>1</sgr:dpSizeNrRegisters>
       <sgr:masterFunctionsSupported>Primitives</sgr:masterFunctionsSupported>
     </sgr:modbusDataPoint>
   </sgr:dpListElement>

<sgr:hexMask> is a hex value and used as bitmask to determine whether a status indicator is set.

The XSD is as follows:

  <complexType name="bitMapType">
    <sequence>
      <element name="mapping" minOccurs="0" maxOccurs="unbounded" type="sgr:bitMappingType"/>
    </sequence>
  </complexType>

  <complexType name="bitMappingType">
    <sequence>
      <element name="literal" minOccurs="1" maxOccurs="1" type="string"/>
      <element name="hexMask" minOccurs="1" maxOccurs="1" type="sgr:hexType"/>
      <element name="boolVal" minOccurs="0" maxOccurs="1" type="boolean" />
    </sequence>
  </complexType>

  <simpleType name="hexType">
    <restriction base="string" >
      <pattern value="[0-9A-F]*"/>
    </restriction>
  </simpleType>

The XSD provides an additional element boolVal. The value is used by the SGrJava implementation to return the status of the respective status indicator (as a result of the 'register-value & mask' operation).

Alternatives

no proposal yet

Acceptance Criteria

Enum's

[ ] The enum 'literal' to 'ordinal' mapping is defined on EI-XML level
[ ] The XSD defines the elements needed to define the enum mapping and a container for the read/written value.
[ ] The SGrJava and the SGrPython libraries support setVal() and getVal() for enums.

Bitmaps

[ ] The status indicator 'literal' to 'bitMask' mapping is defined on EI-XML level
[ ] The XSD defines the elements needed to define a mapping list. The XSD defines a container to for the read write value for each status indicator.
[ ] The SGrJava and SGrPython libraries support setVal() and getVal() for enums.

Open Questions

To be discussed regarding enums within the Java implementation:

The Java implementation supports 2 API methods to set a value:

setVal(String: functionalProfileName, String: dataPointName, String: value)
setValByGDPType(String: functionProfileName, String: dataPointName, SGrGenDataType: value)

For the String value API the usage is very simple and straight forward:

setVal("HeatPumpBase", "HPOpModeCmd", "HP_OFF") where HP_OFF is the enum literal. The validation and conversion of the status literal to the SGrGenDataType is done within the CommHandler library.

For setValByGDPType this is more complex since we need to create a valid instance of SGrGenDatatype to provide it as a value. The complete code is as follows:

SGrGenDataType grGenDataType = V0Factory.eINSTANCE.createSGrGenDataType();
EnumType enumType = V0Factory.eINSTANCE.createEnumType();
EnumMappingType enumMappingType = V0Factory.eINSTANCE.createEnumMappingType();
enumMappingType.setLiteral("HP_OFF");
enumList.setEnumVal(enumMappingType);
sgrGenDataType.setEnum(enumType);
devStiebelISG.setValByGDPType("HeatPumpBase", "HPOpModeCmd", grGenDataType);

Shall we just recommend/encourage using the simple setVal("HeatPumpBase", "HPOpModeCmd", "HP_OFF") ?
Shall we expose a convenience function that builds the SGrGenDataType, including validation (string literal must be one of the defined enums)?

Example convenience function usage:

SGrGenDataType enumValue = devStiebel.buildEnumValueFor("HeatPumpBase", "HPOpModeCmd", "HP_OFF");
devStiebel.setValByGDPType("HeatPumpBase", "HPOpModeCmd", enumValue);

ergozeller commented 1 year ago

See also #86 for input on bitmappers...

ChrisBroenni commented 1 year ago

enums within the Java implementation: much better compared to the legacy version !excellent! enums within C / C++/ C# implementations: no issues expected at a glance

ginoagb commented 1 year ago

I find it a good improvement. Go with it!

Enum's: is this a small error (0,1,3)? should the number be consecutive? Can this be stated in the XSD definition?

ChrisBroenni commented 1 year ago

Enum's: is this a small error (0,1,3)? enumerations may be device dependent.

what I did: I used XXX_NUL if 0 has no meaning. When enums are aligned different (thats most often the case) this can be aligned with a different ordinal lineup in the EID.

In the existing solution I supported re-mapping. This is difficult to understand as it changes with the RW lineup and to put the foot into the trap happens easily.

succestion:

have it aligned "beautiful" at generic layer and use the "real" ordinals at TransportSevice side.

or

use no ordinals generic layer

However, with bitmaps we need the ordinals.

ergo-furrer commented 1 year ago

Enum's: is this a small error (0,1,3)? should the number be consecutive? Can this be stated in the XSD definition?

This is an error. The enum could be returned from the transport service as:

An integer value: consecutive mapping values 0, 1, 2, 3, 4... OR 1, 2, 3, 4...., OR 2, 3, 4, 5..... are valid.
A bit register: 2^X values: Each enum mapping value is defined as a single bit that is set: 1, 2, 4, 8... OR 4, 8, 16, 32... .

I plan to add an additional hexMask to the enum definition to mask out other bits/values, that do not belong to the enumeration, but are part of the same register.

However, the example case 0, 1, 3 .. is not valid since it does not meet the rules given above ( = consecutive int's, or values with just one bit (2^X) exclusively set).

suggestion:

have it aligned "beautiful" at generic layer and use the "real" ordinals at TransportSevice side.

use no ordinals generic layer

My intent is to use the "real" ordinals (TransportService side ordinals) only and map them to string values at the generic side (no artificial beautifully aligned ordinals on generic side).

SmartGridready / SGrSpecifications