Open cbenz opened 1 month ago
Hi @cbenz, thanks for the issue and clear summary!
Could a maintainer of this library tell me whether this is really a new SDMX feature, not implemented yet? […] (I don't find any mention of
PublicationTable
in the SDMX 2.1 information model PDF)
Indeed, I don't find it either: not in the SDMX 2.1 standard you linked, nor in:
So this seems to not be standards-compliant SDMX-ML, idiosyncratic to this data provider. It's possible that the XML returned by this query might pass XML Schema validation (that is, the XML tag is a well-formed <Ref …>
with the expected attributes), but we (and thus the code) have no way of understanding what it is supposed to mean.
Some possibilities:
In any case, the design goals for sdmx1
do not cover non-standard usages. So I think the preferred approach would be to catch this exception, ignore it, and perhaps log a warning message. This would prevent the invalid reference from blocking your use of the other contents of the CategorySchemes. I could take care of making this change.
Meanwhile, could you perhaps please contact the BIS via the means given on their SDMX documentation website, directing them to this issue, and ask them to please clarify what is going on? If it is case (1) above, sdmx1
will eventually need to support this. If (2) or (3), the provider may not be aware that their web service is returning invalid SDMX-ML.
Here's a reply from one Glenn Tice, via the “SDMX User Forum” (linked via the header at https://sdmx.org; unfortunately not public/requires registration):
PublicationTable is one of the Fusion Metadata Registry (FMR) 'extended' structural metadata artefacts. These are not official SDMX artefacts, but follow all the same principles. Publication Tables are 'identifiable' so can be referenced by other structures including Category Schemes as in this case.
In FMR, Publication Tables are 'presentational metadata' objects which describe complex data tables for publication on websites and other media. A full description can be found on the FMR Wiki here. There are examples of materialised publication tables on the BIS Data Portal: Locational banking statistics publication table: BIS,LBS_A1,1.0.
There are no plans to add Publication Tables to the standard SDMX information model. However the SDMX Technical Working Group is examining in general how presentational metadata can be supported, and new official artefacts for that purpose are likely to emerge as a result, maybe in SDMX 3.2.0.
For your use case, a solution may be to modify the code to gracefully ignore references to artefacts with unrecognised packages or classes.
The suggestion aligns with what I wrote earlier (“catch this exception, ignore it, and perhaps log a warning message”). I'll open a PR shortly with these changes and ask for your review.
Running this code:
I get the following stack trace:
I understand that the following XML fragment is not supported, in particular the
class="PublicationTable"
part:This has worked until yesterday. Maybe BIS updated their SDMX API reponse content? Sadly I did not keep a copy of the working response.
Could a maintainer of this library tell me whether this is really a new SDMX feature, not implemented yet?
As of now I don't really get how to add a parser for that, but I'm going to try. (I don't find any mention of
PublicationTable
in the SDMX 2.1 information model PDF)