CDM Serialization Specification

[chrisisla]

Background

To meet its promise of interoperability, CDM requires a defined standard for the exchange of information. Unfortunately, after a detailed analysis, the community determined that the current JSON format had gaps that prevented it from becoming that standard. In November 2023, the SWG approved recommendations made by a TAWG task force for a phased approach to changes to the data standard.

As part of its commitment to promoting the CDM as a standard, ISLA is funding the first phase of this work, working in conjunction with CLOUDRISK, FT Advisory, ISDA and REGnosys. The following provides an overview of the plan for the changes.

The high level goals for this phase of the Serialization project were to focus on:

• interoperability: users of the same version of the model should be able to exchange data regardless of their programming language
• completeness: ability to represent the entire model
• readability: serialized data should be human readible
• compactness: serialized data should be as compact as possible

The design, once accepted by the community, will move into development with a targeted release of CDM version 7.

Principles

When evaluating serialization design alternatives the followed ordered list of principles served as reference:

• object generation: serialized data should facilitate creation of Rune defined objects including enabling the "language's" polymorphic inheritance
• model conformity: to the fullest extent possible, the serialized data should directly conform to the model. The reader has no obligation to keep fields that it does not recognise
• error reporting: report all failures
• atomic types serialisation: data types as dates, times and others should be serialized as far as possible using well established standards/formats as for example ISO.
• ease of integration: to the extent possible, it should be easy to keep serialization in sync with model versions

Some may not be absolute and the final design may have had to make compromises on the extent to which it meets the principles.

Proposal

The CDM object serialization will be enhanced to now include the following new special attributes.

New	Example	Description
@model	"@model": "cdm"	Model short name, located at the top of the JSON
@version	"@version": "1.2.3"	Model version, located at the top of the JSON
@type	"@type": "cdm.event.common.TradeState"	Fully Qualified Type Name, located at the top of the JSON
@data	"@data": "54930084UKLVMY22DS16"	Replaces value annotation
@key	"@key": "abcd1234"	Replaces globalKey metadata annotation
@ref	"@ref": "abcd1234"	Replaces globalReference metadata annotation
@key:external	"@key:external": "my-external-key"	Replaces externalKey metadata annotation
@ref:external	"@ref:external": "my-external-key"	Replaces externalReference metadata annotation
@key:scoped	"@key:scoped": "my-scoped-key"	Replaces scope and location annotations
@ref:scoped	"@ref:scoped": "my-scoped-key"	Replaces scope and address annotations
@scheme	"@scheme": "http://www.fpml.org/coding-scheme/external/iso17442"	Replaces scheme metadata annotation

When serializing:

• special attributes will always be at the top of a node
• null special attributes will no longer be serialized
• internal keys that are not referenced will not be serialized; external keys will always be serialized

When deserializing:

• JSON will be validated for structure and types, not content
• attributes that do not conform to the current version of the model will be discarded

In summary the new serialized format:

• has stronger alignment to model definitions
• improves readability
• is more compact
• provides a more consistent machine-readable representation
• facilitates easier and faster machine processing.

For more details of the proposed structure and processing please refer to the full design document available here: Serialization Specification v0.9.pdf

Feedback

If you have any questions on the design then please add them as new comments to this issue.

[chrisisla]

The design will be presented at the TAWG meeting on November 14th (details of the meeting can be found here: #3212).

The community will then have 3 weeks to review the design and pose any questions or comments that they may have. The expectation is that objections, if any, should be concrete, material and include alternative approaches so that a final review can address the pros and cons.

At the following meeting of the TAWG on December 5th any outstanding questions on the design will be answered, with a view to the final design being accepted at that meeting.

[ahutusoru123]

Question regarding the "@ data" notation - clarified during the call that "@ data" should be used when a primitive value (end leaf) has some meta data attached to it. Conclusion was: adding a larger CDM object example would add clarity on proposed usage.