Some protocols like HKEX OCG have a feature called Field Presence Maps, which allows to skip transmission of unset optional fields.
It's a special type of field which contains a bitmap describing the presence of subsequent fields in the actual message instance. The presence map provides an efficient way to check which optional fields are present in a received message. Bits corresponding to required fields are always set to 1.
Those types may be represented as special datatypes mapped to bitsets. In HKEX OCG, for example, a field of such datatype is included in a message header and also it precedes every repeating group instance:
<datatype name="Bitmap">
<mappedDatatype standard="ISO11404" base="bitstring" element="bit">
<annotation>
<documentation>Field length is a number of octets.</documentation>
</annotation>
</mappedDatatype>
</datatype>
<fields>
<field id="42" name="BodyFieldsPresenceMap" type="Bitmap" implLength="32">
<annotation>
<documentation>Indicates the list of fields that would be present in a message body immediately after this field. Each bit denotes corresponding field's presence.</documentation>
</annotation>
</field>
<field id="101" name="NoEntitlementsBodyFieldPresenceMap" type="Bitmap" implLength="2"/>
</fields>
<groups>
<group id="100" name="Entitlements">
<numInGroup id="100" name="NoEntitlements"/>
<!-- Presence bitmap for this group is the field with id 101 -->
<fieldRef id="101" name="NoEntitlementsBodyFieldPresenceMap"/>
<fieldRef id="102" name="EntitlementType"/>
<fieldRef id="103" name="EntitlementIndicator"/>
<fieldRef id="104" name="EntitlementID"/>
</group>
</groups>
<components>
<component id="1" name="Header">
<!-- ... -->
<fieldRef id="42" name="HeaderFieldPresenceMap"/>
</component>
<!-- Presence bitmap for this component is the field with id 42 -->
<component id="10" name="TestRequestBody">
<fieldRef id="43" name="FirstField"/>
<fieldRef id="44" name="SecondField"/>
<groupRef id="100" name="NoEntitlements"/>
</component>
</components>
<messages>
<message name="TestRequest">
<structure>
<componentRef id="1" name="Header"/>
<componentRef id="10" name="TestRequestBody"/>
<componentRef id="2" name="Trailer"/>
</structure>
</message>
</messages>
There should be a standard approach to associate the bitmap field with the components and groups it relates to, in the same way a data field may be associated with its length field using the lengthId attribute.
Proposal
We recommend adding optional presenceMapId attribute to container elements like message/structure, component, and group. This attribute must reference a field which contains presence flag for every direct member of the container element. This approach is like the one with the lengthId and discriminatorId attributes of a field definition, which are already a part of the standard.
The proposal is to add ability to associate presence map field to the components and groups as discussed in https://github.com/FIXTradingCommunity/fix-orchestra/discussions/197.
Issue
Some protocols like HKEX OCG have a feature called Field Presence Maps, which allows to skip transmission of unset optional fields. It's a special type of field which contains a bitmap describing the presence of subsequent fields in the actual message instance. The presence map provides an efficient way to check which optional fields are present in a received message. Bits corresponding to required fields are always set to 1.
Those types may be represented as special datatypes mapped to bitsets. In HKEX OCG, for example, a field of such datatype is included in a message header and also it precedes every repeating group instance:
There should be a standard approach to associate the bitmap field with the components and groups it relates to, in the same way a data field may be associated with its length field using the
lengthIdattribute.Proposal
We recommend adding optional
presenceMapIdattribute to container elements likemessage/structure,component, andgroup. This attribute must reference a field which contains presence flag for every direct member of the container element. This approach is like the one with thelengthIdanddiscriminatorIdattributes of a field definition, which are already a part of the standard.Example