search

urbanobservatory / standards

Standards and schema documentation for the observatories programme
2 stars 0 forks source link

Proposal: Drop Hydra vocabulary in favour of JSON [Hyper-]Schema #13

Open lukeshope opened 4 years ago

lukeshope commented 4 years ago

Now that I have actually started trying to use some of the things we've been discussing in a real world project, I have concerns that Hydra as a hypermedia doesn't provide enough of the descriptive power we need.

90% of our API functionality is providing access to sensor data through some sensible ontologies, but there are several instances where clients will need to go beyond following links, and instead compose IRIs to describe the filters, constraints, pages, etc. they want.

We know as a minimum that our hypermedia controls need to allow:

I have no doubt more complex scenarios will arise. Especially if people implement API endpoints for sensor management.

To be clear, there is JSON Schema (generally describing the shape of a document), and JSON Hyper-Schema (describing how to interact with an API). I am suggesting implementations MUST use link descriptions (LDO) from Hyper-Schema, but MAY also use bits of JSON Schema (which would be useful for allowing client-side validation for example).

We would use:

Advantages

Disadvantages

Crash course in JSON Hyper-Schema

I've taken an example from the 9.5 Collections in the Hyper-Schema standard, and tried to make it more applicable to us. Assuming a request like GET http://example.webof.uk/sensor/ to obtain a list of all the sensors, the response would look something like:

Link: <http://example.webof.uk/schemas/v1.0.0/sensor-collection.json>; rel="describedBy"

{
  "elements": [
    {"@id": "http://example.webof.uk/sensor/sensor-a"},
    {"@id": "http://example.webof.uk/sensor/sensor-b"}
  ],
  "meta": {
    "current": { "offset": 0, "limit": 2 },
    "next": { "offset": 3, "limit": 2 }
  }
}

The elements in the body could of course contain more properties, or could be left as @id with the onus on the clients to dereference.

The schema to describe the interaction patterns associated with the above document would be obtained with GET http://example.webof.uk/schemas/v1.0.0/sensor-collection.json, with a semantic version number.

{
  "properties": {
    "elements": {
      "type": "array",
      "items": { "$ref": "#/definitions/jsonld-id" }
    },
    "meta": {
      "type": "object",
      "properties": {
        "prev": {"$ref": "#/definitions/pagination"},
        "current": {"$ref": "#/definitions/pagination"},
        "next": {"$ref": "#/definitions/pagination"}
      }
    }
  },
  "links": [
    {
      "rel": "self",
      "href": "sensor{?offset,limit}",
      "templateRequired": ["offset", "limit"],
      "templatePointers": {
        "offset": "/meta/current/offset",
        "limit": "/meta/current/limit"
      },
      "targetSchema": {"$ref": "#"}
    }, {
      "rel": "prev",
      "href": "sensor{?offset,limit}",
      "templateRequired": ["offset", "limit"],
      "templatePointers": {
        "offset": "/meta/prev/offset",
        "limit": "/meta/prev/limit"
      },
      "targetSchema": {"$ref": "#"}
    }, {
      "rel": "next",
      "href": "sensor{?offset,limit}",
      "templateRequired": ["offset", "limit"],
      "templatePointers": {
        "offset": "/meta/next/offset",
        "limit": "/meta/next/limit"
      },
      "targetSchema": {"$ref": "#"}
    }
  ],
  "definitions": {
    "jsonld-id": {
      "type": "object",
      "properties": {
        "@id": { "type": "string", "format": "uri" }
      },
      "required": [ "@id" ]
    },
    "pagination": {
      "type": "object",
      "properties": {
        "offset": { "type": "integer", "minimum": 0, "default": 0 },
        "limit": { "type": "integer", "minimum": 1, "maximum": 100, "default": 10 }
      }
    }
  }
}

This describes all of the links associated with the collection, those being the previous/next/current pages. It also describes the maximum number of items that can be requested per page etc., and the default values. These IRIs for the pages are generate using the IRI templates with pointers to the data within the document, in the meta object.

You can try the two above snippets in the JSON Schema validator.

The client would, with the above two documents, be able to infer the links available, including recognising that prev was absent from the meta object in the document, so no previous page link is available. This would provide the following:

[
  {
    "contextUri": "https://example.webof.uk/sensor",
    "contextPointer": "",
    "rel": "self",
    "targetUri": "https://example.webof.uk/sensor?offset=0&limit=2",
    "attachmentPointer": ""
  },
  {
    "contextUri": "https://example.webof.uk/sensor",
    "contextPointer": "",
    "rel": "next",
    "targetUri": "https://example.webof.uk/sensor?offset=3&limit=2",
    "attachmentPointer": ""
  }
]
SiBell commented 4 years ago

Well the advantages sound good. I'm all for automated documentation. You're not wrong about the learning curve though. There's a series of blogs on https://apisyouwonthate.com that give a good introduction to JSON hyper-schema (Part 1, Part 2, Part 3).

Could do with working through another real-world example, or two, to get a sense of just how easy this is to implement in practise.

It worries me slightly that doca hasn't been updated in a while, nor does it have many weekly downloads from npm in comparison to something like ReDoc.

Also is the line: Link: <http://example.webof.uk/schemas/v1.0.0/sensor-collection.json>; rel="describedBy" actually part of the response body?

lukeshope commented 4 years ago

Absolutely. I will try to hammer out a couple more examples over the coming days, perhaps focusing on our other big hypermedia need: filtering and searching.

I take your point about doca. Instead you could use ReDoc, which uses OpenAPI, though there are a few subtle differences between OpenAPI and JSON Schema. Phil has created a converter. I'm not sure Redoc would pick up the JSON Hyper-Schema elements that describe interactions, probably only the JSON Schema elements that describe validation and structure. Either way, we should be able to put something together that works.

The Link part is a header field in the response sorry, not part of the body.

EttoreHector commented 4 years ago

Guys, thank you so much for this. You are surely a couple of levels above me when it comes to linked-data technologies and standards. A couple of examples that relate with typical use-cases in the observatories would be really great!

lukeshope commented 4 years ago

Cheers, I am working on some examples and some tooling. Draft PR is #16 where I've tried to explain what I'm doing and why. More to follow next week!

:-)

SiBell commented 4 years ago

I've been looking into this a bit more. My thoughts: