search

NASA-PDS / registry-api

Web API service for the PDS Registry, providing the implementation of the PDS Search API (https://github.com/nasa-pds/pds-api) for the PDS Registry.
https://nasa-pds.github.io/pds-api
Other
2 stars 5 forks source link

As a user, I want the API response media types to be compliant with RFC 6838 #464

Closed jordanpadams closed 2 years ago

jordanpadams commented 2 years ago

💪 Motivation

...so that we are in spec with expected OpenAPI requirements.

📖 Additional Details

Not sure if we need to do anything or what the RFC actually requires, but per SwaggerHub, it looks like we are supposed to do something like vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml?

The Earth CMR API uses response formats like application/vnd.nasa.cmr.umm_results+json and application/vnd.nasa.cmr.umm_results+json; version=N.N.N. But then there are also media types like application/atom+xml and application/echo10+xml. Maybe we can just register our media types somewhere?

First task of this ticket is to investigate what may or may not need to be done.

⚖️ Acceptance Criteria

Given a deployed server When I perform a request on any end-point using header Accept:application/vnd.nasa.pds.pds4+xml Then I expect to get a result in format pds4+xml

Same for Accept values: application/vnd.nasa.pds.pds4+json

The test can be found in postman collection https://github.com/NASA-PDS/registry/blob/main/docker/postman/postman_collection.json (use application postman https://www.postman.com/downloads/)

⚙️ Engineering Details

jordanpadams commented 2 years ago

@tloubrieu-jpl FYI ☝️ not sure what or if we need to do anything here, but thought it was at least worth documenting as a requirement and we can research more details.

al-niessner commented 2 years ago

@tloubrieu-jpl @jordanpadams

It will take a while to grok the RFC, but the immediate thing that jumps out is that using 'x.' might be the simplest option as it only requires agreement between the writer and reader of the output. All other options seem to require a more formal registration process to share with the whole world. Deciding if you want to do the formal registration or limit the non registered formats to 'x.' is a good start.

tloubrieu-jpl commented 2 years ago

Hi @jordanpadams ,

The form to register mime type is there: https://www.iana.org/form/media-types

from what I quickly read that can take some time and we would need to read all the RFCs. Maybe someone at JPL or PDS can help us with the process. But it sounds like that would make sense for the pds4+xml since we can consider ourself as a 'standards-related organization'.

But I don't think that would work for the pds4+json unless it is included in the scope of the DDWG. The we should use the vendor tree, which is not bad since we will be under a nasa umbrella.

I keep note of the version attribute which I like for the future compliance management. I've seen that if can also be a way of 'flavouring' the standard mime types for example application/xml schema=pds4, but I prefer the registered mime type or vendor tree options for that usage.

jordanpadams commented 2 years ago

@tloubrieu-jpl @al-niessner I think we go with the vendor tree option and be done with it. Not sure the bang is worth the big to go through all the effort to create the new media types just so it's "better". Something we can maybe pursue way down the road when the API is stabilized and more widely used.

The XML that is included in PDS4 metadata labels is a managed, international standard we should probably register. I think application/pds4+xml is not quite since it is a wrapped version of the PDS4 metadata (including registry-related metadata). So I think going the vendor tree route is the best bet.

How do these sound?

vnd.nasa.pds.pds4+json vnd.nasa.pds.pds4+xml

This keeps the door open to add other custom formats as we would like:

vnd.nasa.pds.votable vnd.nasa.pds.pdap vnd.nasa.pds.epn-tap

tloubrieu-jpl commented 2 years ago

That sounds good to me !

Good point that pds4+xml is node exactly pds4 since there is some wrapping, note however that for single product response pds4+xml should match exactly with pds4 standard.

jordanpadams commented 2 years ago

@tloubrieu-jpl is that true though? What about the meta information?

al-niessner commented 2 years ago

@tloubrieu-jpl @jordanpadams

Can easily do the vendor tree in swagger yaml. Filling out the request form is far more complicated and requires a dedicated individual to have their email recorded at iana.org to answer questions about it. Maybe a pds.mime.subtype.standards@jpl.nasa.gov that provides a fixed address that can then be distributed to a team member on PDS?

Also, to register them a public standard is required. Is there a JSON standard for pds4 or did we just approximate one from the XML standard? If there is not a JSON standard, then what is the process for creating one?

Just for fun, I searched to see if votable was already a mime subtype. It is not. However, votable is a standard that google found immediately and it claims that for its mime type to use:

Note that in RFC 6836 section 3.4 it states 'x-' has been deprecated and to use 'x.' making votable specification a bit antiquated.

The second point votables brings up is versioning. Do we accept version parameters for the mime type like 1.1, 1.2, or 1.3 for votables? How about the 17 versions of pds4? pds3? Yeah, I saw pds3 on the website so somebody in the world is going to ask for it when they see it.

What is the advantage of being registered? There seem to be only two effects to being registered:

  1. an email address to contact -- that cynical me thinks is probably dead before the registration process is even finished.
  2. a 50 click chipmunk trail -- thin, lightly used, and hard to follow -- to the specification, google is much faster even for the pds4, pdap, and epn-tap standards.

So why vendor tree over unregistered tree if swagger does both? Did 5 second test and swagger is happy with 'x.'. Is it political or want to be like the cool kids or a bad requirement -- another way of saying political I guess -- or something else entirely?

Side note: it would only take a day or two to do application/x.votable.

I guess another option would be implement it all as 'x.' then as you get them registered change them in swagger and RequestAndResponseContext. Also gives you the advantage of asking for forgiveness with 'x.' and register only those that cannot be forgiven -- meaning less work.

tloubrieu-jpl commented 2 years ago

Thanks @al-niessner for looking at these details. That is useful.

I was not aware of votable but after a quick google that sound like a good example to follow. However, from the RFC section you linked above, it sounds like, the vendor tree should be preferred over the x. tree.

I think that as dev team we should not invest ourselves in the IANA registration process, the DDWG should do that for us if need be.

I would stick to the previous conclusion of this ticket to use the vendor tree. Is swagger also happy with the vnd. prefix ?

al-niessner commented 2 years ago

@tloubrieu-jpl

Sorry, but we still have a contradiction. The main goal of the ticket is to make the mime types RFC 6838 compliant. Using a vendor tree without registration is just as non-compliant as what we have now. They only way to comply with this ticket on compliance is to use 'x.' until registration takes place then swap over.

Or change the ticket to use vendor tree despite non-compliance with RFC 6838.

jordanpadams commented 2 years ago

@al-niessner @tloubrieu-jpl per discussion going ahead with:

vnd.nasa.pds.pds4+json vnd.nasa.pds.pds4+xml

@jordanpadams to register these