Clarification on Usage of AMQP 1.0 Headers for Content-Type and Encoding

[timburke]

Goal

The goal of this issue to help ensure that the CFX standard is able to support future message serialization formats and compression types easily without a danger to any existing client that may not support them. In order to enable this, there needs to be a clear way for a client to know the chosen serialization and compression format before trying to decode a message.

Overview

Currently the CFX SDK sends CFX messages serialized as JSON objects with optional deflate compression. The CFX standard calls out that future versions may support different (potentially nonjson) serialization formats and likely more compression algorithms on top of those underlying serialization formats.

Without a clear way to obtain the chosen compression/serialization formats from a received cfx message itself, it could be very difficult to safely allow multiple formats to coexist on the same broker since clients would have to guess what combination of serialization/compression was used for a given message.

AMQP 1.0 has standard properties to include these two values (content-type and encoding) in a globally understood way. Setting both properties now to valid MIME types will help ensure that future additions of other choices for either serialization format or compression will not negatively impact existing clients since they will be able to inspect the message properties and know how to decode it.

Details

The AMQP 1.0 protocol has a standardized way of communicating both serialization format using the content-type property and optional compression using the content-encoding property.

AMQP 1.0 defines that both content-type and content-encoding properties, when set should be valid MIME content-types.

Currently, the CFX SDK does not set content-type explicitly and sets content-encoding to a non-null value only when compressing, in which case it uses a nonstandard value CFX-COMPRESSED.

See: https://github.com/IPCConnectedFactoryExchange/CFX/blob/master/CFX/Transport/AmqpUtilities.cs#L66

The AMQP 1.0 standard specifies that content-encoding must be an IANA registered MIME type (see amqp 1.0 standard page 78).

Recommendations

The following is a list of potential ways to address the above described issue for consideration and disuccsion.

1. Add a content-type header on all messages with MIME type application/json
2. When compressing messages, change nonstandard MIME type from CFX-COMPRESSED to, for example, application/x-gzip.
3. In the SDK, check content-type and content-encoding and validate their contents before decoding. This will allow clients to operate in a mixed encoding environment with clear error messages when receiving a message they are not able to decode (due to unknown serialization format or compression type)
4. In the SDK accept an empty content-type header as a synonym for application/json to allow full backwards compatibility with legacy senders that do not set content-type correctly.
5. In the SDK accept an value of CFX-COMPRESSED as a synonym for no-compression to allow full backwards compatibility with legacy senders that send a nonstandard content-encoding value. See https://github.com/IPCConnectedFactoryExchange/CFX/blob/master/CFX/Transport/AmqpUtilities.cs#L123

References

• AMQP 1.0 Standard: http://www.amqp.org/sites/amqp.org/files/amqp.pdf

[syrandel]

Thank you Tim. I approve your recommendations and join myself to you. The number of issues opened and not answered is growing. Can someone from the initial team having designed CFX be involved here ?

[simon-smith]

Hi guys. Thank you for your comments. The CFX team are meeting on the 19th. I will raise these issues/suggestions during the meeting.