MetadataStructure Reference in DSD must allow using nested MSD metadata attributes

[dosse]

The description of the "Metadata" element in the DataStructureType object currently says: "... Note that the referenced metadata set cannot contain nested metadata attributes, as these are not supported in the data. ...".

Using nested metadata attributes within a DSD is an absolutely required feature, and this can well be supported in data messages. Also nested Categories can be used in Categorisations, so there doesn't seem to be a reason, that this cannot be supported.

[sdmx3mdt]

@dosse I assume this refers to the XML schemas. Grateful if you could confirm.

[dosse]

@sdmx3mdt I saw this in the schemas but I cannot exclude that it was not somewhere in the IM also. But I guess that J would know from where this text originates.

[sdmx3mdt]

Possible impact on the Information Model and Section 2 - TF2 to assess.

[glenntav]

For reference, Section 2 DRAFT 1.0 (approved draft for public review) states:

5.5 Reference Metadata in Data Structure Definition and Dataset An important change of SDMX 3.0 is the ability to reference an MSD within a DSD, in order to report any Metadata Attributes defined in the former to Datasets of the latter. This is achieved by the following: -In a DSD, the user may add a reference to one MSD. -In the Attribute Descriptor of the DSD, the user may include any Metadata Attributes defined in the linked MSD. --For each link to a Metadata Attribute, an Attribute Relationship may be specified (similarly to that for Data Attributes). -In any Dataset complying with this DSD, Metadata Attributes may be reported according to the specified Attribute Relationship. --Any hierarchy of the Metadata Attributes defined in the MSD is ignored; Metadata Attributes are reported as Data Attributes. -In Data Constraints, the user is allowed to restrict values for Metadata Attributes, in the same way as Data Attributes (more on this in section “9 Constraints”).

[dosse]

@glennmdt It might be stated there but it doesn't practically work and thus represents a design error that has been overseen in the draft. The MSD allows, for instance, defining an attribute "NAME" as child of an attribute "CONTACT" and then another attribute "NAME" as child of "COLLECTING_ORG" since attribute IDs do not have to be unique. Therefore the DSD can only use them if it prefixes the attributes with their parents, e.g. "CONTACT.NAME" and "COLLECTING_ORG.NAME" like it's easily done with categories in categorisations. Also, I do not see why this has been restricted. The idea of linking MSDs to DSDs was not to make it a second class metadata management. We need to respond to the underlying needs, otherwise the new model will again be without much use like in 1.0, 2.0 and 2.1.

[sdmx3mdt]

27 July 2021 Approved in principle by SDMX TWG subject to 1 week written consultation with the TWG/SWG membership closing on 04 August.

TWG Members consulting on proposed changes using written procedure with deadline of 04 August 2021 for comments or objections.

Details of the proposed changes are in the attached document: SDMX TWG TF2 - public review issues #14 - MetadataStructure Reference in DSD must allow using nested MSD metadata attributes.docx

[sdmx3mdt]

No comments or objections received from TWG members in response to the written procedure consultation closing on 04 August 2021. Change Approved.

Outstanding actions:

• SDMX-ML-schemas team - update XML schemas to reflect the revised approach as detailed in the change request document.
• SDMX-JSON / CSV schemas team - update the JSON and CSV schemas / specifications.
• TF2 - update Section 2 in particular paragraph 5.5 which states that any hierarchy of metadata attributes is ignored.

[sdmx3mdt]

Done:

• SDMX-JSON / CSV schemas team - update the JSON and CSV schemas / specifications.

Outstanding actions:

• SDMX-ML-schemas team - update XML schemas to reflect the revised approach as detailed in the change request document.
• TF2 - update Section 2 in particular paragraph 5.5 which states that any hierarchy of metadata attributes is ignored.

[sdmx3mdt]

SDMX-ML schemas: Schemas updated to allow metadatasets to be included within data messages at the obs, series, group and dataset levels.

Section 2: Section 2 corrected to state that an unrestricted MetadataStructureDefinition can be attached to a DSD for the purposes of reporting reference metadata against data.