JSON:API support - Githubissues

gabesullice commented 6 years ago

This is a really interesting client. I love it!

Are there any plans to add support JSON:API? If so, would you consider it (or a PR that does)?

evert commented 6 years ago

Hi @gabesullice ,

I think JSON:API support would be really great! I didn't realize they had links front and center as much as they did, so it seems like an almost obvious fit.

evert commented 6 years ago

The part that will be harder to express, is things inside the data property. From the example on the website it looks like it's just a list of embedded resources, but what relationship do they have with the top-level resource?

gabesullice commented 6 years ago

data always contains 1 or more items that represent the primary objects for the requested resource.

For example, if you had a resource /posts, data would be an JSON array containing representations of posts.

If you had a resource /posts/1, data would be a JSON object of a representation of the post with id 1 or data would be null if the response status is a 404 Not Found.

Other sibling keys of data are jsonapi which contains API version info, errors which might contain error info, and links which obviously contains hypermedia links like self, next and prev (amongst others).

The most interesting of the sibling keys is included which might contain representations of entities on the server related to the "primary" data. E.g. a request to /posts?include=author would have representations of posts under data and representations of those posts' authors under included.

I think for Ketting, you're right, the most important thing would be how to handle this structure elegantly:

GET /jsonapi/posts

{
  "data": [{
   "attributes": {"title": "Foo"},
   "links": {
     "self": {
       "href": "/posts/1"
     }
   }
  }, {
   "attributes": {"title": "Bar"},
   "links": {
     "self": {
       "href": "/posts/2"
     }
   }
  }],
  "links": {
    "self": {
      "href": "/posts"
    }
  }
}

How would one "follow" these links given that they're contextualized by their location in the response body?

One idea I had a second argument to follow() named context. This context would be a selector applicable to the underlying media type (e.g. a CSS selector for HTML or JSON Path for JSON).

I think this fits well with the web linking spec, given it's language about "context IRIs", even though I don't think it was intended to be used this way.

gabesullice commented 6 years ago

Practically, that might look like this:

let resource = ketting.getResource('/posts');
let post = resource.follow('self', '.data[1]').get();
console.log(post.attributes.title); // "Bar"

evert commented 6 years ago

included would be very easy to deal with, because it's a direct analogue to HAL's _embedded.

What will be nice for JSON:API users is that they can completely ignore included items and not care if they are there or not. If they are, the cache will be prepopulated with them.

I'm going a little bit through the specification, and noticed that data is basically a 'single resource' or 'an array of resources'.

Would something like this be possible

For the 'single' case, merge the resource links with the links of the top-level document.
For the 'multiple' case, add a pretend link (such as json-api-resource) and treat the items in the array as embedded/included resources?

Then your example might look like:

const resource = ketting.getResource('/posts');
const posts = await resource.followAll('json-api-resource');
for(const post of posts) {
  console.log(await post.get());
}

gabesullice commented 6 years ago

included would be very easy to deal with, because it's a direct analogue to HAL's _embedded.

👍

What will be nice for JSON:API users is that they can completely ignore included items and not care if they are there or not. If they are, the cache will be prepopulated with them.

Yesss. That's awesome.

For the 'single' case, merge the resource links with the links of the top-level document.

I think for the vast majority of standard cases, this would be fine. But I don't think it would be an assumption that would hold up in all cases.

Just recently, I've been implementing a way to get versioned objects from a JSON:API server. It's possible to end up with something like this:

GET /post/1?resource_version=rel:latest-version

{
  "data": {
    "type": "post",
    "id": 1,
    "links": {
      "self": "/post/1?resource_version=id:2"
    }
  },
  "links": {
    "self": "/post/1?resource_version=rel:latest-version"
  }
}

At a later point, if you refreshed, the data might have a query string of ?resource_version=id:3 in it.

gabesullice commented 6 years ago

For the 'multiple' case, add a pretend link (such as json-api-resource) and treat the items in the array as embedded/included resources?

I don't think that would not be possible. But I don't know that I really love the idea either. I think I'm having a knee-jerk reaction to the "magic" of it. It also feels like a case where the public API is being driven by the internal implementation of the client rather than the most intuitive thing for the user (which is not to say that my suggestion is the most intuitive either).

gabesullice commented 6 years ago

Totally unrelated to this issue, but I'm not sure where else to put it...

I just shamelessly stalked your Github and blog (😅) and came across your post about http2 and APIs and after reading it, I bet you might be interested in this little experiment of mine just for fun: https://github.com/gabesullice/hades.

evert commented 6 years ago

That's super interesting. I've been running around an idea for a last couple of weeks to try and write a RFC-style standard for something likeyour X-Push-Please. Except, it would take a relationship.

I get the hesitance against the json-api-resource magic. The suggestion came from the fact that I feel that resources in a collection should be expressed as some link with a relationship type.

Another option might be to treat data items in a JSON:API resource as a sort of sub-resource with their own #fragment. But then the question remains, how do you get a sub-resource from a resource, if not with a relationship type.

evert commented 6 years ago

What I'm really also saying is that it's kind of unfortunate that JSON:API didn't model collection resources as their own resource + a bunch of item relationships =)

gabesullice commented 6 years ago

That's super interesting. I've been running around an idea for a last couple of weeks to try and write a RFC-style standard for something likeyour X-Push-Please. Except, it would take a relationship.

Please let me know if you start on it, I'd love to contribute!

Another option might be to treat data items in a JSON:API resource as a sort of sub-resource with their own #fragment. But then the question remains, how do you get a sub-resource from a resource, if not with a relationship type. ... What I'm really also saying is that it's kind of unfortunate that JSON:API didn't model collection resources as their own resource + a bunch of item relationships =)

Totally agree on the last point.

In our server implementation, we're thinking about doing something in the same spirit for included objects.

Honestly, JSON:API has a... complicated relationship with good hypermedia practices. I'm sure there will be more things to consider, even if this gets solved.

I just went to read 8288§3.2 Link Context, which linked me to RFC3986§5 Reference Resolution. I admit I haven't really completely read/grokked it, but it looks like it has some related concepts and might be important for Ketting in other ways.

Link headers look to have been designed to work with hierarchical/nested data since they can be anchored to resource fragments (The anchor example and explanation in 8288§3.5 was helpful to me). Given that Link headers can have different context from the base resource too, maybe that gives some more weight to my idea about a second parameter to follow(), since you might want to follow a link with the same relation type but a different anchor.

A last point about JSON:API linking wonkiness before I sign off for the day:

There are yet more link sub-contexts within a resource object:

"data": {
  "type": "post",
  "id": 1,
  "relationships": {
    "author": {
      "links": {
        "self": "/posts/1/relationships/author",
        "related": "/posts/1/author",
      }
    },
    "tags: {
      "links": {
        "self": "/posts/1/relationships/tags",
        "related": "/posts/1/tags",
      }
    }
  }
}

In that case, I think yet another implicit relation type would be needed, but it wouldn't be as simple as just using item or json-api-resource because the links objects are under named keys, not in a simple array.

wimleers commented 5 years ago

Exciting discussion here! I think it'd be interesting to get JSON:API spec maintainers @dgeb and @ethanresnick to chime in here :)

evert commented 5 years ago

Your explanation makes a lot of sense. I wasn't aware of anchor and it's having me buzzing with ideas a little bit. It also addresses a frustration I had with HAL (which is that it's not really possible to have anything besides top-level links).

I left some comments and thoughts on your PR (#111).

ethanresnick commented 5 years ago

As a JSON:API editor (who's thought a lot about hypermedia), I'm happy to weigh in. I hardly have the full context around ketting and the various available options, but I can say a bit about hypermedia in JSON:API.

Honestly, JSON:API has a... complicated relationship with good hypermedia practices.

This is an understatement :) But hopefully JSON:API's model will get simpler at some point, and we do know know about most of the issues raised in this thread (see e.g. https://github.com/json-api/json-api/issues/898, https://github.com/json-api/json-api/pull/834, and https://github.com/json-api/json-api/issues/913).

For now, here's how I would probably model JSON:API documents as collections of RFC 8288 links:

The top-level links key is straightforward: the context uri is just the request uri and the link relation is the key name. Getting the target URI(s) is a bit of a pain, because JSON:API has so many formats that the values in a links object can take on (i.e., null,string, string[], { href: string }, { href: string }[]), but what to do is easy conceptually.
For all other links objects, establish a context URI through the value of the self link in that object. If there is no self link, give up trying to extract that object's links. (You could try to come up with a fragment that could be used to create a context uri, but json doesn't have an official fragment format afik, and a sensible fragment format for json:api would probably be something based on json:api's type–id identification scheme anyway, rather than, say, json pointer. Moreover, I think it's totally reasonable to require that people using JSON:API for hypermedia to provide a self link.) Then, the representation for the resource identified by the self link is the whole object that contains the links object.
When data contains an array of json:api resource objects, add a collection link from each resource object that points back to the request URI, and a set of item links from the request uri to the self uri of each item in the data array.

So, taking (a simplified version of) the example document from JSON:API's homepage:

{
  "links": {
    "self": "http://example.com/articles",
    "next": "http://example.com/articles?page[offset]=1"
  },
  "data": [{
    "type": "articles",
    "id": "1",
    "attributes": {
      "title": "JSON:API paints my bikeshed!"
    },
    "relationships": {
      "author": {
        "links": {
          "self": "http://example.com/articles/1/relationships/author",
          "related": "http://example.com/articles/1/author"
        },
        "data": { "type": "people", "id": "9" }
      }
    },
    "links": {
      "self": "http://example.com/articles/1"
    }
  }],
  "included": [{
    "type": "people",
    "id": "9",
    "attributes": {
      "firstName": "Dan",
      "lastName": "Gebhardt",
      "twitter": "dgeb"
    },
    "links": {
      "self": "http://example.com/people/9"
    }
  }]
}

You'd end up with links:

context	rel	target
http://example.com/articles	self	http://example.com/articles
http://example.com/articles	next	http://example.com/articles?page[offset]=1
http://example.com/articles	item	http://example.com/articles/1
http://example.com/articles/1	collection	http://example.com/articles
http://example.com/articles/1	self	http://example.com/articles/1
http://example.com/articles/1/relationships/author	self	http://example.com/articles/1/relationships/author
http://example.com/articles/1/relationships/author	related	http://example.com/articles/1/author
http://example.com/people/9	self	http://example.com/people/9

Note: there are no item + collection links for the data within relationships objects because of https://github.com/json-api/json-api/issues/913.

In terms of cached representations, you'd end up with:

`http://example.com/articles`

the whole response document

`http://example.com/articles/1`

{
    "type": "articles",
    "id": "1",
    "attributes": {
      "title": "JSON:API paints my bikeshed!"
    },
    "relationships": {
      "author": {
        "links": {
          "self": "http://example.com/articles/1/relationships/author",
          "related": "http://example.com/articles/1/author"
        },
        "data": { "type": "people", "id": "9" }
      }
    },
    "links": {
      "self": "http://example.com/articles/1"
    }
  }

`http://example.com/articles/1/relationships/author`

{
  "links": {
    "self": "http://example.com/articles/1/relationships/author",
    "related": "http://example.com/articles/1/author"
  },
  "data": { "type": "people", "id": "9" }
}

`http://example.com/people/9`

{
  "type": "people",
  "id": "9",
  "attributes": {
    "firstName": "Dan",
    "lastName": "Gebhardt",
    "twitter": "dgeb"
  },
  "links": {
    "self": "http://example.com/people/9"
  }
}

I hope that helps in some way! Lmk if there's anything else I can add.

evert commented 5 years ago

@ethanresnick , thanks for this explanation. @gabesullice, does that alter your perspective on this? My take is that only links blocks with self links should be considered.

What's not clear to me yet is, if other links blocks appear in the response body and they do have a self link, is there an implicit relationship between those (for the lack of a better term) sub-resources and the resource in which they appear in?

ethanresnick commented 5 years ago

thanks for this explanation

Sure :)

To your question:

What's not clear to me yet is, if other links blocks appear in the response body and they do have a self link, is there an implicit relationship between those (for the lack of a better term) sub-resources and the resource in which they appear in?

Are you talking about the implicit collection/item links I added? If so, I would say those are justified because the spec is pretty clear that, when data is an array, it's because the response is a collection of resources. Then, clearly, each entry is an item in that collection. Specifically, the spec says (emphasis added):

Primary data MUST be... an array of resource objects, an array of resource identifier objects, or an empty array ([]), for requests that target resource collections

Also/unrelated, I realized shortly after posting that the method I gave for extracting the representations of the "sub-resources" is actually a little broken. When the "subresource" is a relationship object like you'd find at http://example.com/articles/1/relationships/author, then my earlier method works. But, when the subresource is a "resource object" in JSON:API speak, as in http://example.com/people/9, you actually have to take the value I described above and wrap it in a data key. So the representation becomes:

{ 
  "data": {
    "type": "people",
    "id": "9",
    "attributes": { /* ... */ },
    "links": {
      "self": "http://example.com/people/9"
    }
  }
}

I would also probably repeat data.links at the top-level (for reasons you'll see below), so the full representation would be:

{ 
  // simply a copy of data.links
  "links": {
    "self": "http://example.com/people/9"
  },
  "data": {
    "type": "people",
    "id": "9",
    "attributes": { /* ... */ },
    "links": {
      "self": "http://example.com/people/9"
    }
  }
}

Sorry about that omission.

The one final caveat about extracting "sub resource" representations in this way is that, with the introduction of profiles in 1.1, there are now top-level links that implicitly apply to every sub-resource too, namely profile links. If a representation has a top-level profile link, it means (among other things) "the linked profile describes some extra meaning associated with this document's members", but those members could be in the subresources. So, a complete method for extracting representations of the subresources would involve cascading the profile links down. So, the representation for http://example.com/people/9 would be:

{
  "links": { 
    "profile": [/* links from top-level links.profile in http://example.com/articles */],
    // other copied links from data.links
  },
  "data": { /* same contents as `data` in example above */ }
}

Cascading these profile links certainly isn't hard, but it's a bit annoying that you'd have to add a special case for it (since cascading top-level links from the containing resource in the general case certainly is not safe). It also wouldn't be very future proof: if a new top-level link was added later that also implicitly applied to subresources, existing code would miss it.

A much better approach imo would be for the JSON:API spec to allow the sub resources in the original response to simply repeat the top-level links that apply to them. So, a response for /articles could be:

{
  "links": {
    "self": "http://example.com/articles",
    "next": "http://example.com/articles?page[offset]=1",
    "profile": ["http://example.com/some-profile"]
  },
  "data": [{
    "type": "articles",
    "id": "1",
    "attributes": { /* ... */ },
    "relationships": {
      "author": {
        "links": {
          "self": "http://example.com/articles/1/relationships/author",
          // profile repeated here
          "profile": ["http://example.com/some-profile"]
        },
        "data": { "type": "people", "id": "9" }
      }
    },
    "links": {
      "self": "http://example.com/articles/1"
      // profile repeated here too
      "profile": ["http://example.com/some-profile"]
    }
  }]
}

Then, the subresource extraction logic can be simpler and relatively future proof. To summarize, it would go like this:

if the subresource is a relationship object, pull out the whole object containing the links key and use that as the representation (as in my original post)
if the subresource is a resource object, pull out the whole object containing the links key, place that object as the value of the data key in a new object, and add a links key to that new object that has the same content as data.links.

So, the representation for /articles/1 (given the response above with the repeated links) would be:

{
  // this is just a full copy of data.links; no special-casing required.
  "links": {
    "self": "http://example.com/articles/1"
    "profile": ["http://example.com/some-profile"]
  },
  data: {
    "type": "articles",
    "id": "1",
    "attributes": { /* ... */ },
    "relationships": {
      "author": {
        "links": {
          "self": "http://example.com/articles/1/relationships/author",
          "related": "http://example.com/articles/1/author",
          "profile": ["http://example.com/some-profile"]
        },
        "data": { "type": "people", "id": "9" }
      }
    },
    "links": {
      "self": "http://example.com/articles/1"
      "profile": ["http://example.com/some-profile"]
    }
  }

That works pretty nicely imo. To make it happen, though, you (or @gabesullice) would have to open an issue/PR on the JSON:API repo allowing the repetition of the profile links like this -- and maybe jumpstarting a more general discussion about the identification/extraction of subresources in a JSON:API document.

evert commented 5 years ago

Hi @ethanresnick ,

When I originally read your comment I read the whole thing. But when I looked at it yesterday I did a bit of a poor job with skimming it. All your answers make perfect sense, and so does the item / collection relationship.

I feel with this in hand there's already a lot of stuff I can implement. Curious what @gabesullice thinks

gabesullice commented 5 years ago

What we're trying to work out is: can we programmatically map a resource object to a resource on the client, and if so, how?

FWIW, we're certainly not the first to encounter this JSON:API stumbling block.

@steveklabnik summed up the distinction between a resource and a resource object (a.k.a "entity") nicely and I think that's where the confusion is coming from here.

@ethanresnick addressed a lot of my concern about treating resource objects in a collection as an independent resource. I.e., that you need to "wrap" it in the JSON:API data envelope and that you have to be concerned about how the envelope contextualizes the resource object(s) within it (e.g. via a profile links). I think cascading links is a nice heuristic for doing that, but I'm afraid it could be prone to error and lost information (what about the meta or jsonapi members?) IOW, I think @ethanresnick is right about "maybe jumpstarting a more general discussion about the identification/extraction of subresources in a JSON:API document". I agree with that completely.

@evert, I haven't looked at you PR yet (I'll do that next), but I think you're primarily concerned with this because you want to pre-populate a cache of the target entities to avoid an additional request when "following" items? Honestly, I don't think that needs to happen (at least not soon). Following top-level links alone is fine to begin with.

You could try to come up with a fragment that could be used to create a context uri, but json doesn't have an official fragment format afik, and a sensible fragment format for json:api would probably be something based on json:api's type–id identification scheme anyway, rather than, say, json pointer.

I think a JSON:API profile would be a great way to establish a fragment format ;) *I don't really agree that type-id is a good scheme (because of relationship objects and the difficulty of parsing them out), but we can have that discussion elsewhere!

I think using the self link to establish a context URI is a good start, but I'm afraid it will suffer from the same pitfalls mentioned above in that it will lose contextual information that it inherits from the top-level document (a.k.a "envelope").

evert commented 5 years ago

@evert, I haven't looked at you PR yet (I'll do that next), but I think you're primarily concerned with this because you want to pre-populate a cache of the target entities to avoid an additional request when "following" items? Honestly, I don't think that needs to happen (at least not soon). Following top-level links alone is fine to begin with.

There's a few different bits here that all are somewhat related.

First, there's implicit link relationships through the item reltype. I want to extract those too. This isn't very difficult, and allows parent->followAll('item').
Then, there is indeed caching. This is bonus, but not strictly needed. The drawback of not having this, is that you might end up doing more requests than you need.
Kind of related to both. If a collection has items, and those items have their own relationships. The 'correct' way to get access to those is calling:

parent.follow('item').follow('some-other-rel');

This works without caching the caching bit, but without caching it does imply that, in order to get the some-other-rel resource, we do need do an extra HTTP request, even though that information would already have been available in the parent request.

This is not a deal-breaker for me, but I imagine someone using this library with JSON:API might be surprised that there are more HTTP requests than needed.

So ultimately, if there is a sensible JSON:API way to (as you say) map a Resource Object to a real Resource, you get a lot for free.

gabesullice commented 5 years ago

First, there's implicit link relationships through the item reltype. I want to extract those too. This isn't very difficult, and allows parent->followAll('item').

I think the rule should be:

if a resource object under a data array (it has to be an array) has a self link, it implicitly becomes an item link belonging to the top-level links object.

To start, I would issue an actual request in order to follow that link. I would not emulate it until the mapping rules are more fully understood.

Edit: Also, I would not bother with item links for relationships either.

evert commented 5 years ago

So the bits i have implemented now are links for top-level objects and treating collection-members as 'item' links (if they have a self link).

Is there more that can be done today? I'm thinking it might be possible to parse relationships for non-collection resources and treat members as related links, but I'm not that sure how useful that is given that we can't map include resource objects to full resources.

evert commented 5 years ago

Although if someone is interested build a generic HATEAOS api browser based on Ketting it might be a nice little bonus.

ethanresnick commented 5 years ago

I think cascading links is a nice heuristic for doing that, but I'm afraid it could be prone to error and lost information (what about the meta or jsonapi members?)

+1 to this concern. and to maybe holding off on trying to synthesize representations until the rules are better understood.

Besides that, I'm excited to the see the progress here with the latest PR!

evert commented 5 years ago

Thanks all for the contributions. I'm closing this ticket for now. Hopefully any future gaps can be closed through an ext iteration of JSON:API. Happy NY!

badgateway / ketting

JSON:API support #109

`http://example.com/articles`

`http://example.com/articles/1`

`http://example.com/articles/1/relationships/author`

`http://example.com/people/9`