Create rudimentary HTML media type for NLDI resources.

[dblodgett-usgs]

In the spirit of the W3C data on the web best practices, as much as possible, NLDI resources should be available in an html content type. These pages can be extremely rudimentary -- For resources that are ostensibly hypermedia, the html format would display links that allow a person to traverse to/from nested and parent resources. For GeoJSON resources, an as-simple-as-possible GeoJSON page such as is implemented by pyGeoAPI e.g. https://info.geoconnex.us/collections/hu10/items/ should be supported.

MVP here is intentionally barebones as this is primarily intended for a developer audience.

[kkehl-usgs]

Is this for flowlines (https://labs.waterdata.usgs.gov/api/nldi/linked-data/nwissite/USGS-05429700/navigate/UM?distance=1) or features (https://labs.waterdata.usgs.gov/api/nldi/linked-data/nwissite/USGS-05429700/) or both? For flowlines, I think I'll need to modify the API to return more info (name, uri) because right now that API is only returning the nhdplus_comid.

[dblodgett-usgs]

Both. If it's free to add some additional attributes to the flowline response, I think it would be wise to go ahead and do it. I would start small. GNIS Name, GNIS ID, StreamOrder, feature-URL would be the obvious ones.

If it's work to add them, I don't think it would be required in this ticket to add the additional attributes.

[kkehl-usgs]

Would the HTML responses be on different endpoints? Something like?

linked-data/{featureSource}/{featureID}/navigate/{navigationMode} (GeoJSON) linked-data/{featureSource}/{featureID}/navigate/{navigationMode}/html (HTML)

[dblodgett-usgs]

No -- they should be the same path.

However, I would like to support three schemes for media-type negotiation.

linked-data/{featureSource}/{featureID}/navigate/{navigationMode} + content negotiation by http headers.

or

linked-data/{featureSource}/{featureID}/navigate/{navigationMode}.{html|json|jsonld}

or

linked-data/{featureSource}/{featureID}/navigate/{navigationMode}?f={html|json|jsonld}

I could be talked out ot supporting the .{html|json|jsonld} but for sure want to support the ?f= pattern and traditional content negotation.

[dsteinich]

Spring is discouraging the use of the . notation and has deprecated code related to it - https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#mvc-ann-requestmapping-rfd

[kkehl-usgs]

I think we need a separate issue for jsonld support. Work to support html is enough for its own ticket, which is already in JIRA with 5 story points.

[dblodgett-usgs]

Totally agree on not adding jsonld -- that was more leading than it was meant to have implications for this issue. I have no problem just supporting the ?f= content negotiation override if spring doesn't support/discourages the. form.

[dsteinich]

I strongly disagree with implementing this issue as part of the data service. Especially if 'MVP here is intentionally barebones as this is primarily intended for a developer audience.' In my opinion, this violates the separation we have had between data services and user interfaces.

OpenAPI is a semi reasonable exception to the rule in that we do not maintain the UI, and are merely including a third party tool to accomplish the functionality. In reality, it could also be hosted from a different deployment and not included in the data services jar.

OpenAPI also allows for the exploration of the API and data returned, meeting the goal of a minimal UI for developers to explore the API.

[dblodgett-usgs]

This requirement is associated with W3C Data on the Web and Spatial Data on the Web best practices.

These two core best practices demand a human-readable representation of metadata and data on the web.

• https://www.w3.org/TR/dwbp/#ProvideMetadata
• https://www.w3.org/TR/sdw-bp/#indexable-by-search-engines

While OpenAPI provides one part of this, the resources an API represents also must be interpretable by both machines and human users. This is why OGC APIs are using both OpenAPI and traversable hypermedia/hypertext for API documentation.

If this should be implemented as a separate module/component/(insert the right words here) that calls the data service, I'd be fine with that. Maybe a third component could implement the content negotiation logic that layers on top.

[jkreft-usgs]

yeah, this is where I both agree with these approaches (there should be an HTML representation of the data that we produce) and disagree (keeping the API code pure data significantly reduces scope creep. Making and maintaining multiple HTML representations of this data makes me tired). If there is going to be HTML representation in the API, we need to be very careful about it (and be prepared to push back hard on stakeholders who get excited about it and want to keep adding more features to what are supposed to be minimal pages). There is also a potential bomb to throw in that any HTML that we create is supposed to follow the US Web design system standards, and these will not, at least at first. We will need to be cognizant of that as we move forward.

[dblodgett-usgs]

I agree on all fronts. These HTML representations make this a self describing API that doesn't bomb people's browsers and that's it.

I'd go as far as to say the default html media type could be an "unimplemented" statement for browser clients, but when a client asks for an html media type with content negotiation (e.g. from a browser) we have to send something other than JSON.

That's why we've gone in the direction of the html media type being a link to the JSON media type. Any additions will be to describe the API and maybe to render the JSON contents as HTML.

[dblodgett-usgs]

Let's table this for now. It's clear that we lack consensus on public API design and there is no sense in forcing something forward that people are uncomfortable with. I'll leave this issue open but remove it from the V release.

[dblodgett-usgs]

I talked through this with Jim today and I think we have a good compromise.

There are some competing demands here.

1. Adding a lot of code for something that is really just convenience for a few people is not worth the cruft.
2. We may want these resources to have a full-blown html representation in the future, but that is not clear right now.
3. The only major issue is that browser clients can be blown up by the JSON media type download.

The compromise is -- for the feature collection view of a feature source and for the navigation results -- to return what is essentially an error page if and only if the client requests text/html and does not request application/json. The response can be as simple as "An HTML representation is not available for this resource" -- but a link to the JSON resource as has already been implemented would be ok too as long as the amount of code added isn't unreasonable from a project maintenance point of view.

This will ensure we don't have big database hits and don't blow up people's browsers when people click through the JSON links to documents that really aren't intended for a browser.

[kkehl-usgs]

Dave Steinich said if we go down this road, he'd like me to refactor my original PR so that there is only one HTML endpoint for all the controllers and it returns a generic page. This minimizes the impact to the existing code. I will do that.

[dblodgett-usgs]

If there's any way to have the endpoint remain the same ... i.e. the client thinks it's getting an html page at the requested url, that would be preferable to having some kind of redirect to a different url.

[dblodgett-usgs]

I'm getting a 406 error here: https://labs-dev.wma.chs.usgs.gov/api/nldi/linked-data

The links from html to json are http:// and should be https://

Otherwise, looks good.

[dblodgett-usgs]

With latest release to dev, this works as expected.