Composites reusing other types

[mjpt777]

If I define a reusable type such as a Boolean enum then how can I reuse that across multiple composites?

[donmendelson]

Possible solutions:

1. Use a <type> element within the <composite> with name attribute matching an <enum> defined outside the composite.
2. Create a new element <refType> with just a name attribute for the same purpose. The referenced type name would be an <enum> or other encoding type. (Inline definition as with the current schema would still be allowed.)

Either way, it should allow offset attribute to satisfy issue #11.

[mjpt777]

1. Feels like a hack and not discoverable. <type> is defined as a encodedDataType which is not an enum, set, or other type.
2. <refType> will not pass schema validation so is unworkable.

Can you provide examples of how it was tested for the specification?

[donmendelson]

I understand that this does not work with the current schema; I'm trying to throw out ideas about how to resolve it. I'm open to other proposals.

[mjpt777]

Maybe something like <ref name="" type="" offset=""/>

[mjpt777]

For reference I see people want to define a common type that is part of a message. This common type can then be passed to methods for getting or putting fields and used across multiple message types. This is a good reason to complex composites, but importantly they need to refer to other common types.

For example, common fields in different order type messages can be pulled out and handled by common code otherwise the code must be repeated when no common type exists.

[mjpt777]

We would also like to implement the composite features this week before the holidays are upon us. Do you think you can get an answer from the committee soon?

[donmendelson]

The type attribute would be the name of the <enum> or other encoding type (analogous to XSD "ref" attribute), while name attribute would tell its usage within the composite.

A contrived example with the referenced type is Boolean enum:

<composite name=futuresPrice">
  <type name="mantissa" primitiveType="int64" />
  <type name="exponent" primitiveType="int8" />
  <ref name="isSettlement" type="boolEnum" />
</composite>

[donmendelson]

@mjpt777 a block of common fields could be defined as a <group> even if the it is not expected to be used as repeating instances. I will add that as a separate issue.

[mjpt777]

@donmendelson The <group> tag is not common across messages so I don't see how that can be used as a structural type in a code base. Am I missing something?

[mjpt777]

Our choice will be implement as RC3 but this is not really workable for composites to be generally useful or go with some assumptions. We could assume offset and ref gets approved but will not have time to go back and change the reference implementation if a different decision is made.

Maybe a good lesson from other standards bodies is not to publish a spec until a working reference implementation has been tried.

[jimnup]

We haven't published a specification yet. We have release candidates.

On Mon, Dec 14, 2015 at 11:09 AM, Martin Thompson notifications@github.com wrote:

Our choice will be implement as RC3 but this is not really workable for composites to be generally useful or go with some assumptions. We could assume offset and ref gets approved but will not have time to go back and change the reference implementation if a different decision is made.

Maybe a good lesson from other standards bodies is not to publish a spec until a working reference implementation has been tried.

— Reply to this email directly or view it on GitHub https://github.com/FIXTradingCommunity/fix-simple-binary-encoding/issues/12#issuecomment-164478026 .

[mjpt777]

@jimnup It's the same. It is published. See RFC :-)

Specs need to be verified with implementation as primary input.

[kleihan]

It is a chicken and egg problem. Release Candidates are the deliverables FIX chose to be able to make changes prior to a (Draft) Standard that can only be subject to very minor changes. Engagement within the FIX membership was quite low, hence the decision to move to github with the latest Release Candidates. Without "publishing" it there we cannot get any feedback from the wider community to make changes or enhancements before defining it to be a (Draft) Standard. We should not get hung up on the definition of "publish". Fact of the matter is that changes and enhancements are currently still possible and will be made. At the same time it should not be a moving target, i.e. a decision should be made rather sooner than later to make RC4 the last Release Candidate of V1.0 which can be the basis for the Draft Standard after the public review period of 90 days. The Draft Standard for Version 1.0 will be accompanied by announcements from FIX, much more than any of the Release Candidates, due to the commitment from FIX to move any further changes or enhancements to higher versions, protecting any investments made. Before removing "Draft" FIX wants to see two interoperable implementations, verifying the paper spec.

[mjpt777]

@kleihan How does the FIX standards body prove, in a formal way, that two implementations interoperate correctly? For this, other standards implement a TCK (Technology Compatibility Tootkit).

I understand the chicken and egg issue and many standards bodies have struggled with it. The one major lesson is that a spec needs to be led by trying to implement. Otherwise specs can become unworkable. Look to the history of the CORBA persistence service to see how this can fail without implementation.

The IETF get a lot of this right https://www.ietf.org/about/standards-process.html

The goals of the Internet Standards Process are:

• technical excellence;
• prior implementation and testing;
• clear, concise, and easily understood documentation;
• openness and fairness; and
• timeliness.

Note the first two points are led by technical implementation. This tends to catch many errors and flaws early.

https://datatracker.ietf.org/doc/rfc2026/?include_text=1

[kleihan]

@mjpt777 I agree that technical implementation is key. I do not think the intention is to formally prove 100% interoperability of all SBE features. It is also a question of effort needed and bandwidth available. FIX is to be fit for purpose, i.e. it will likely be more of a judgement call of experts that show interest. This may not be enough for an IETF standard but it is probably the best we can do right now. The AMQP standard showed interoperability by having prototype implementations from two vendors exchange messages with each other.The same could be done for SBE with pre-defined FIX messages encoded and decoded with different implementations.