XML as an API data format

[pmackay]

I would appreciate any input on this. If a stakeholder requests it as an supported format for an API, is that reasonable? How many APIs are being developed that include XML? Or is it considered legacy and clients should simply update to be able to process JSON?

[rossjones]

Warning: contains some anecdata. Disclaimer: I actually like XML as well as JSON

I think it is entirely reasonable to ask for XML, under some circumstances. TLDR; business-users sometimes prefer XML.

• Tooling - JSON is great for providing raw data, with some limited structure and very limited typing, but the tooling for it is still not great. There is a lot of tooling available for XML that can make it appealing for some use-case (XSLT, XQuery, XPath, XSD - Transform, Query, Navigation, Schema validation). Not every user of the data will be accessing it from a dynamic language.
• Typing - In situations where typing and structure are important, XML can actually be useful for validation. Particularly in cases where the data you are returning has a good fit with a particular schema, for instance XBRL (which HMRC only started supporting 3 years ago - so not really legacy ;) ).
• When users are outside our bubble - We’re in a bubble (web) inside a bubble (non-commercial) and sometimes it is hard to see how things we prefer are used, or not, elsewhere. XML is still very widely used in the commercial sector, anecdotally the 2 customers I had when having a DGU break were both very heavy XML users.
• Standards - XML is standardised, afaict RFC7159 is still only a proposed standard.

[philandstuff]

XML isn't so much a format, it's a language for building formats: XHTML, Atom, RDF/XML, SAML, etc. I think it's often clearer to talk about which specific XML dialect you're talking about.

One concern I have with XML is that it's so complex that it's difficult to do correctly. This applies double for security concerns such as XML signing and encryption (see also #6). That said I can't imagine backing a "standard" that "banned" XML wholesale.

[adamkcooper]

I agree. XML is a language the real question is what's the protocol / usage.

We shouldn't rule out XML but we should have some PRI principles on usage. On 19 Feb 2016 10:06, "Philip Potter" notifications@github.com wrote:

XML isn't so much a format, it's a language for building formats: XHTML, Atom, RDF/XML, SAML, etc. I think it's often clearer to talk about which specific XML dialect you're talking about.

One concern I have with XML is that it's so complex that it's difficult to do correctly. This applies double for security concerns such as XML signing and encryption (see also #6 https://github.com/alphagov/tech-and-data-standards/issues/6). That said I can't imagine backing a "standard" that "banned" XML wholesale.

— Reply to this email directly or view it on GitHub https://github.com/alphagov/tech-and-data-standards/issues/16#issuecomment-186147802 .

[psd]

We can provide generic XML formats such as Atom, KML, RDF/XML etc, and a service, or indeed anybody can generate other domain-specific formats using register data, but for registers we need to understand the user need for the format before supporting it, as it's hard to deprecate them.

[andy-gray]

TBH though JSON may be the toy-du-jour, we should have exactly the same concerns about language vs format, and "what's the protocol" with it as well.

[PlanetCJ]

XML is very verbose but the key reason why we still use it as our primary data exchange format is because we can apply a schema to it. There's no accepted schema format for JSON. Its very important that when we pass data messages from one service to another that the 'sending' service is as happy as it can be that the data conforms to the corporate XML standard. The consumer also checks that the data it retrieves conforms to this standard as part of its service wrapper. We keep the message sizes down by using consumer-driven contracts and tolerant reader patterns.

[ghost]

Where I work, we use XML message payloads in SOAP based web services all the time. Even on new services. When we discuss interface API with external 3rd parties XML is usually the most commonly offered option. Typically always validated against .xsd schema. I get comfort in using SOAP/XML as it has a rich tool set and standards that are still widely used as 'rossjones' mentions above. However I also like REST/JSON for internal API's as they are quicker to pull together. We are recently considering publishing both XML & JSON API's. The coding tech thats available these days make it almost trivial to consume either XML, JSON, YAML, etc, its really does not matter too much. What I'm hoping will come from this site is some finer guidance on the construction of API's. In the absence of such I Google best practices. Living in hope of some good clear standards to emerge - if JSON is king then please define example structures to allow wider usage and clear understanding. Same for SOAP/XML.

[edent]

Quoting from the GDS API Guide

Offer content for each thing as human-readable HTML, with links to content in alternative machine-readable representations, like:

• JSON for convenient processing in most programming languages
• JSONP and JSON with CORS for client-side JavaScript
• CSV for importing into spreadsheets
• Atom for feeds

All text should be encoded as Unicode, following the Government Digital Service (GDS) Cross platform character encoding profile. Where possible also offer other formats most suited to a specific domain, like:

• iCalendar for events
• vCard for names and addresses
• KML and geoRSS for geographical data
• m3u for playlists

[chris48s]

There are also a non-zero number of edge cases and inconsistencies around JSON parsing: http://seriot.ch/parsing_json.php which are worth being aware of.

[Lawrence-G]

XML is acceptable as part of an existing standard