How to interpret presence at the field level?

[karypid]

Hello,

Again more of a question as I try to understand the specification.

According to the XSD for SBE 1.0, you can specify presence at both the encoding and the field level:

    <xs:complexType name="encodedDataType" mixed="true">
...
                <xs:attributeGroup ref="sbe:presenceAttributes"/>
...
    <xs:complexType name="fieldType">
...
        <xs:attributeGroup ref="sbe:presenceAttributes"/>
...

Consider the following type definitions:

        <type name="f_int16_optional" primitiveType="int16" minValue="10" maxValue="20" presence="optional" nullValue="42" semanticType="All attributes in use" />
        <type name="f_uint16_constant" primitiveType="uint16" presence="constant" semanticType="Constant value 123">123</type>
        <type name="f_int16_required" primitiveType="int16" minValue="100" maxValue="200" semanticType="Required within range" />

Now consider a message with these 3 fields which use them, which alter the presence the field level:

        <field name="f_int16_optional_2_required_override" id="1" type="f_int16_optional" presence="required" />
        <field name="f_uint16_constant_override" id="2" type="f_uint16_constant" presence="required" />
        <field name="f_int16_required_2_optional_override" id="3" type="f_int16_required" presence="optional" nullValue="0">1</field>

I then have the following questions:

1) Since f_int16_optional_2_required_override is no longer optional, how should the application interpret the null value "42" for this field? Is 42 now within the valid values range, so it is ok to send/receive and process it?

2) Since f_uint16_constant_override is no longer constant, it needs to be encoded on the wire and the block length of the message should expect an extra 2 bytes in this case? I am asking because in that case, I suspect SBE tool may have a bug here (for me it does not generate a setter for the field and the block length is 2 less than expected).

3) Since f_int16_required_2_optional_override is now optional, how do I set the null value to what I want? SBE tool ignores the zero in the nullValue attribute and the 1 in the tag content so it may have a bug there?

I looked at the documentation under Field Encoding and found that:

a) In the field presence section:

By default, fields are assumed to be required in a message. However, fields may be specified as optional. To indicate that a value is not set, a special null indicator value is sent on the wire. The null value varies according to data type and encoding. Global defaults for null value may be overridden in a message schema by explicitly specifying the value that indicates nullness.

This indicates in my example f_int16_required_2_optional_override should allow me to set the null value there which is not what I experience.

b) In the Common field schema attributes it seems that all encoding attributes (min/max/null/ presence) are at the field level? Or is that referring to types? The schema makes it clear that message fields can only have presence.

[donmendelson]

In SBE version 1.0, presence attribute was allowed on either on a field, or as a convenience, on a type that is reused for many fields. However, this was not an override model. Rather, if the presence is given at both levels, they must be in agreement. In section 4, "presence mismatch" is listed as an error.

If` the attribute is specified on both a field and the encoding that it references, the values must be identical.

This model was considered confusing, so in version 2.0 the schema was changed to support presence on the field level only; fields no longer inherit presence from their encoding types. (Members of composite types can still have constant presence, however.)

[karypid]

Just started looking at 2.0 spec. I see it is RC1 status. Any sense of how long before this is promoted to final? I would rather use this directly, provided it's just a matter of a few months...

[karypid]

Also, I don't see anything in section 4 of 1.0 specification regarding minValue/maxValue/nullValue mismatch, but I see that in version 2 they have also been moved to field-level only (or in composite type). Are these 3 supposed to be handled the same way in version 1? (so, you should not allow a field to modify them if they are present at the type level - they can only be the same).

Thanks

[donmendelson]

To answer your question about version 2.0 RC1 status there is no set deadline for publishing the release candidate; it will be a collective decision of the FIX High Performance Working Group (participants welcome). Once an RC is published, there is a period of public review. When the working group is satisfied with an RC, it is recommended for promotion to Draft Standard, and eventually to a final Technical Standard.

See Technical Standard Proposal Process

[donmendelson]

You are correct that version 1.0 is silent on minValue/maxValue/nullValue mismatch and the issue is moot in version 2.0. Your interpretation seems reasonable to me.

[karypid]

Thank you.

I have decided to target 2.0 in my work and just try to keep up-to-date with the work in progress.