What's different/better/worse than other JSON hypermedia media types?

[emmanuel]

Some sort of comparison with other well-known JSON media types would really help me evaluate which of these I should use for a given service.

HAL+JSON (Hypertext Application Language) is the oldest of these of which I'm aware (originally published in mid-2011). Its author Mike Kelly has also produced a corresponding XML media type.

Collection+JSON is detailed in the book Building Hypermedia APIs with HTML5 and Node, by Mike Amundsen. I don't think there is a corresponding XML media type.

Finally, Percolator.js for Node.js has a sort-of/kind-of media type in its default production of JSON responses.

What's better/worse/different about Siren than any of the above?

[gpittau]

what really matters is what makes them incompatible with each other... I mean: if I can implement a layer for HAL resource consumption, and, I can then replace that by one which consumes siren, from my client perspective both media-types are compatibles. Now, if HAL does not provide forms or X does not provide embedded links, my client can not be abstracted from the representation and will need to be too smart.

...thinking in terms of HFactors

[paulwalker]

I picked up @mamund's book (the physical edition even!) about a year ago and finally cracked it open as I've recently just started a new project. And now I've stumbled upon what appears to be the canonical discussion comparing the various hyper media types of out there from the authors themselves. While the actual topic may have been whether an attempt should be made to unify, it's certainly served as an informative comparison to me at least :-).

The funny thing is, last time I started a JSON API from scratch five years ago...I sprinkled my JSON with meta-data so very similar to what @kevinswiber has defined here with Siren...I used an actions object with key/value objects to describe the various API resources right down to fields including validation data like min and max fields.

Looking for a standard for this sort of thing a few years back, the closest I could see was what Google started doing with their discovery apis based upon JSON schema: http://tools.ietf.org/html/draft-zyp-json-schema-04. My question for you guys (including you @mikekelly) is whether or not you considered JSON schema to solve some of the same problems that you are now using your respective media types for? For the last few years, I have considered the use of JSON schema in some fashion as the preferred approach to building discoverable, connected/linked JSON APIs, but now I'm obviously starting to question that.

[paulwalker]

Also, FWIW, in my experience the metadata in my JSON paid immediate dividends in human consumption with both web and mobile teams furiously consuming the functionality. Right away the developers saw the value in parsing urls from a hash rather than hard coding them and eventually generic client code sprouted from using the field data for validation to a full on generic library that for non form based functionality with a UI test harness to boot. I never realized generic client state transition workflows or anything from it, but that's certainly interesting if perhaps overly ambitious.

[zdne]

generic client state transition workflows

That. Generic client = state machine interface.

(sorry for hijacking the topic :)

[kevinswiber]

@paulwalker Yes, I've looked at JSON Schema, but I think it's solving for a different (though related) problem space.

The obvious fit is in using it to describe the shape of request or response bodies.

How do you see it being used to augment existing media types?

[kevinswiber]

generic client state transition workflows That. Generic client = state machine interface.

(sorry for hijacking the topic :)

I'm currently working on a project using Siren to model resources as state machines. Representations reflect the current state and available transitions. This is for an open source Internet of Things platform. This happens to be a really great place for hypermedia, because devices-as-state-machines is a great model.

I'm looking forward to exploring more in this space.

(@zdne, I know you know this already. Just informing others. ;)

[paulwalker]

Hi Kevin, Thanks for the response, I hope you are enjoying your Memorial day weekend.

Right, the most obvious overlap is in the description of the accepted request bodies as described in Siren's actions. My initial thoughts were that the use of JSON schema was more appropriate than HTML5 input types, but I do see the value of a more rich set for browser based javascript clients (a use case I certainly have an interest in). While it may not map exactly to the capabilities of other clients, it is more descriptive than JSON schema's seven primitive types and I certainly can envision a mobile or desktop clients taking advantage of this.

Additionally, since I posted my original question I have now come across JSON Hyper-Schema. Obviously the overlap of problem space is much greater between Siren and JSON Hyper-Schema, so I'd like to include the authors @kriszyp @garycourt @geraintluff in case they have some thoughts.

There are other elements of JSON schema/hyper-schema I can see value with:

• URI Templating (Siren provides the oft used rel next/previous examples, but how to describe a URI to arbitrary page in a list?)
• URI-Reference, relative URIs, URI resolution
• targetSchema (i find the possibility that other media types may be described as possible responses intriguing. input: crop dimensions, output: re-cropped image)

@kevinswiber, not trying to stir up a hornet's nest...at the end of the day, I simply want to develop a JSON API that robustly (completely?) describes it's own input/output :-).

[apsoto]

A bit off topic, but FWIW I've been using a JSON schema against a Siren API in my CI build just to make sure I don't break backwards compatibility (removing a property, action, etc).

I guess my point is that Siren and JSON Schema can be complementary.

[paulwalker]

@apsoto Not at all off topic. I'd like to do the same sort of thing for input validation on the API. And I'd love if the stack I used did it for me! I feel we're not far from that, but it all starts with a reasonable http media type/; foo/whatever; format=/getshitdone so we can all get busy creating this sort of stuff.

How do you generate/maintain your schema (vs spec tests)? Wouldn't it be nice if you didn't have to define all this in multiple places?

[geraintluff]

I agree, JSON (Hyper-)Schema and Siren look similar in many ways. Both specify client-server interactions and describe links between items, et cetera. I'm not completely neutral on this, and I apologise if I've misunderstood some fundamental things about Siren, but here are my impressions:

The biggest difference I can see is that Siren is "in-band" - a given piece of data can be described using Siren, and possible interactions for it can be known once the data is received. With JSON Schema, it's more like OO programming - a JSON Schema describes a general category of data. The idea is that you can have very concise data instances, which you combine with the (referenced) schema to infer links/interactions.

There are various form-generation tools for JSON Schema, so you can still generate UIs/clients, but its strength is API description. The ahead-of-time nature of this description means that as well as validation (server-side and client-side), you can auto-generate documentation - or even client libraries in various languages.

(A secondary observation is that Siren seems to focus more on presentation (e.g. there are input types like "datetime" or "reset" that map very intuitively onto HTML form controls). JSON Schema is more focused on the data model - you can say a lot of the same stuff (e.g. {"format": "date-time"}) as well as arbitrarily complex input structures, but controls like "reset" that don't produce any data aren't covered.)

[paulwalker]

Well, the field type definitions are lifted directly from html5 input types so it's a direct mapping. I suspect the inclusion of the reset type was not intentional as submit was purposely removed.

[apsoto]

@paulwalker I hand rolled the schemas and then wrote a rails integration spec test with a custom matcher to validate an api response against a json schema. I'm using the json schema ruby gem.

Hope this snippet below can point you in the right direction.

    JSON.parse(response.body).should match_json_schema "user"
    JSON.parse(response.body).should match_json_schema JSON.parse(schema_string)
    JSON.parse(response.body).should match_json_schema File.read("someschema.json")

RSpec::Matchers.define :match_json_schema do |expected_schema|
    match do |actual_json|
      require 'json-schema'
      expected_schema_hash = JSONValidate.local_schema_for(expected_schema)
      @errors = JSON::Validator.fully_validate(expected_schema_hash, actual_json)
      @errors.nil? || @errors.length == 0
    end
end

[kevinswiber]

@kevinswiber, not trying to stir up a hornet's nest...at the end of the day, I simply want to develop a JSON API that robustly (completely?) describes it's own input/output :-).

@paulwalker I happen to be one of the few who believes a place exists for all this technology. In fact, I think it's great. We can't advance if we live in an isolated land of theory without branching out to the broader world of practice. The way to do that is by sharing our thoughts and building software.

It's true; the features you listed do not exist in Siren. However, I do not see how JSON Hyper-Schema and Siren are even competing. Can they be complementary? Or is the entire premise of JSON Hyper-Schema that all affordances belong in a metadata document? If it's the latter, I disagree with that for several reasons, but I'll save 'em for now. :)

[kevinswiber]

@geraintluff

Thanks for joining the party. :)

The biggest difference I can see is that Siren is "in-band" - a given piece of data can be described using Siren, and possible interactions for it can be known once the data is received. With JSON Schema, it's more like OO programming - a JSON Schema describes a general category of data. The idea is that you can have very concise data instances, which you combine with the (referenced) schema to infer links/interactions.

Yes, this is true. The design is intentional. Siren believes the availability of state transitions is a decision made at runtime by the server. The ability to insert any number of hidden fields in a transition is a way to maintain statelessness, as well, even through server evolutions.

I'm not making a judgement call of "right" or "wrong," but I do believe the design goals differ.

(A secondary observation is that Siren seems to focus more on presentation (e.g. there are input types like "datetime" or "reset" that map very intuitively onto HTML form controls). JSON Schema is more focused on the data model - you can say a lot of the same stuff (e.g. {"format": "date-time"}) as well as arbitrarily complex input structures, but controls like "reset" that don't produce any data aren't covered.)

Siren borrows HTML5 input type definitions. The older input types are unfortunately named, even for HTML. If you read the descriptions of these input types in the HTML specification, such as checkbox, you will see no mention of "square boxes that show a checkmark when selected." The definitions do not focus on the presentation.

In practice, I often decompose complex input structures into simpler action fields. This reduction of data shape complexity will not always be an option, but it has worked for me thus far. It actually makes me revisit my design when I encounter a use case like this. That said, Siren documents could be extended to allow "schema" attributes on input fields. We don't have an official model for extensions (or a way to broadcast their existence in a representation) at this point, but it's something we've talked about on the Issues list and on the Google Group.

[kevinswiber]

@paulwalker I hand rolled the schemas and then wrote a rails integration spec test with a custom matcher to validate an api response against a json schema. I'm using the json schema ruby gem.

@apsoto I think this is a great way to use JSON Schema with Siren. Thanks for sharing.

megathread

[ericelliott]

I'm using JSON-Schema to validate Siren payloads on the server side. I do see a little overlap, but my use is mostly complementary.

[ericelliott]

Another note - many of JSON-Schema's validation rules can be automatically converted to Siren's HTML5-based form field validations.

[paulwalker]

the features you listed do not exist in Siren However, I do not see how JSON Hyper-Schema and Siren are even competing

Yep, not seeking a vs battle, can we grow Siren though? Let's start with relative URIs at least?

[faassen]

Anyone have any opinions on JSON-LD and Hydra specifically? JSON-LD has some overlap with JSON Schema, and obviously Hydra overlaps with a lot of hypermedia efforts.

[kevinswiber]

@faassen This thread is too long to remember everything, and it's quite a commitment to re-read. :)

I think JSON-LD and Hydra are pretty cool, though I still need to spend more time with them. I learned a lot more about Hydra from @lanthaler at the recent API Craft Conference.

Using JSON-LD to describe the shape of API data is a great concept if that's what you need.

I believe operations in Hydra need to be defined up front to adhere to a certain schema. If this is true, and I'm sure Markus will correct me if I'm wrong, it doesn't meet my personal requirement of having available transitions be dynamic and defined by the server at runtime.

In general, I see value in all these different media types. In this volatile time, I think it does make sense to keep researching and keep trying new things. I won't be hindered by a sense of pride in what I've created, because software is not that simple. Sometimes it's A. Sometimes it's B. Sometimes it's 42.

[faassen]

The thread was very long and interesting for me to read.

I've been spending time with JSON-LD. I like especially the tools for expansion and compaction based on context it offers. I've started tinkering with other tools you could build on top of that.

I've also been spending time with Hydra. I have more trouble wrapping my head around Hydra, in part because the spec isn't finished and assumes a bit more semantic web knowledge than I have.

If by "operations defined up front" you mean operations like POST, PUT and the like, Hydra allows that to be dynamic. Operations can either come from the resource's representation, so if I understand your use case right you can change what particular schema they accept.

For operations attached to the external "API documentation" part of Hydra it is different. Hydra allows the API documentation to change dynamically during runtime. This is currently rather casually mentioned in the draft spec (only show particular operations in the API doc if the user has permissions), which set me off on a long discussion with @lanthaler about how this doesn't fit my notion of "API documentation", and how the client is even to know the API documentation has changed. So I think it's part of the intent of Hydra to allow this kind of dynamism, even though it's too dynamic for me.

Now Hydra can solve this in another way by changing the types of the resources. You could for instance have a "post_foos" type that allows a POST operation of schema foo, and a "post_bars" that allows a POST operation of schema bar. Since this doesn't require a reloading of the API documentation at unpredictable times I'd rather it talked about that approach more in the spec.

You got me wondering now what happens if Hydra sees a resource with both types. Does it accept both schemas? I think so, but the spec needs to become more explicit about that too.

[lanthaler]

I believe operations in Hydra need to be defined up front to adhere to a certain schema. If this is true, and I'm sure Markus will correct me if I'm wrong

Yep, this is wrong :-) Hydra allows you to either define operations upfront in what we call an ApiDocumentation or embed them directly in the responses returned by the server. Have a look at these slides, they illustrate both:

http://www.slideshare.net/lanthaler/creating-awesome-web-apis-is-a-breeze/37

(Slide 43 obviously just shows a snippet of the complete ApiDocumentation)