REST API Design

[jbrisbin]

Need to come up with a REST API design is that respectful of the hypermedia-based approach to HTTP resources (HATEOS) but is extensible enough to encompass support for OData as well.

[markpollack]

WRT to commit https://github.com/SpringSource/spring-data-services/commit/bfd5449375148d0e1f05021eed954d3475c5b1e3

As this is for Spring Data Repositories, why the reference to 'service', in particular in the usage {"link": {"rel": "service",...

Shouldn't this be prefaced by saying, assume PersonRepository, AccountRepository Spring Data Repositories exist. Then a request without an accept header to the root will return

HTTP/1.1 200 OK
Date: ...
Server: Spring Data Services Web Exporter/1.0.0.BUILD-SNAPSHOT

http://localhost:8080/baseurl/person
http://localhost:8080/baseurl/account

and with an Accept Header something other than "rel:" "service" would be used...?

At a high level i think the mapping looks good but we need to drill in on details. The discussions I remember floating around were

1. A get on /baseurl/person/1 would return the non-collection backed fields of the domain object marshalled into JSON but any relationships would be represented as links in the body (not header - also not a fan of putting it in the header). Basically a discussion on what our 'default' representation should be with the assumption that the primary consumer of the representation is the browser via a javascript library like jquery.
2. A put on /baseurl/person/1 would not allow only the update of the entity corresponding to the ID=1. If the JSON body had ID=2 that would be an error and the update would fail.

Jon, can you add a separate section for action under /baseurl/domain/id. for example, what are required formats, what error codes can be returned for each request.

[jbrisbin]

I'm trying to keep with the notion of "services" because that's what we're calling this stuff. I don't want to talk about "Repositories" at the top level because that just happens to be the first implementation of a WebExporter (name TBD).

"person" in the URL corresponds to nothing in reality. It's simply a unique key to a service implementation that defines the behavior of the endpoint. In the case of a RepositoryWebExporter (name TBD), it corresponds to the lowercased part before the word "Repository" in the short class name. But that's an implementation detail specific to the repository implementation and not a fundamental part of the WebExporter in general.

So generating a link to rel: "repository" wouldn't be semantically correct because it's not a reference to a repository but to a service. Generating a link to rel: "resource" would be correct for the links included in the body of the document.

Re: The ID stuff: I'd say including a property in the document that corresponds to the entity's ID property should be ignored. I say ignored rather than an error because some mappers in the client side (like ExtJS DataStore) will want an ID property to be present in the document and will not function correctly if we force an error if the value is included as a property on the document. We should simply ignore the value coming in in the property and use the one from the URL.

[markpollack]

I think the connection is not very deep to implementing services. the URL http://localhost:8080/baseurl/service1/ could be anything of course, but what would /baseurl/domain/id mean for a service?

The JSON representations for doing create, update, delete, interacting with repositories will be quite different than for interacting with services. I think there will be more overlap for invoking custom finders, /baseurl/domain/query?=byNameAndAge, but even there there can be extra arguments we need to handle that are repository specific, e.g. sort/page etc.

Let's drill down the the JSON representation for rows #2 and #3 in the table. That is creating a new entity and getting the list of available resources (assuming we so the paging like amazon does). Then the get/put/delete actions in row #3. It can be based on using a PersonRepository where Person has a 1-many relationship to Accounts.

I really have no idea what are valid values to associate with 'link', service just feels a bit off to me, curious as to other opinions.

Sounds reasonable to me to ignore the incoming entity ID contained in the json body.

[jbrisbin]

Just pushed commit 1584c26512, which updates the proposed REST API document.

[markpollack]

Silly things, the .md file gets cut off on the right hand side and for some reason the html file doesn't get rendered as HTML even when I ask to view the 'raw' file via the github link. Am I missing some sorta setting to view the .html as rendered HTML on github?

[jbrisbin]

That's why I added the HTML file in the same directory. Download it and view it in your browser offline.

There's no good way I could find to circumvent the Github styling so that the table is usable by looking at it on Github.

----- Original Message -----

Silly things, the .md file gets cut off on the right hand side and for some reason the html file doesn't get rendered as HTML even when I ask to view the 'raw' file via the github link. Am I missing some sorta setting to view the .html as rendered HTML on github?

---

Reply to this email directly or view it on GitHub: https://github.com/SpringSource/spring-data-services/issues/1#issuecomment-3634576

[poutsma]

I just read the docs, and overall I like it. I definitely think we're on the right path here.

I do have some comments/questions:

• As Mark mentioned earlier, I'd like to see a little bit of introduction as to what the underlying data model is/repositories are. Just a couple of lines, with perhaps some pseudocode. I think that makes it easier to understand.
• I think the use of $ref (which i take it comes from Referencing in Dojo ) could use a reference and perhaps some explanation as to why you chose to use this way of linking.
• For partial updates, you might want to consider using PATCH instead of PUT. Spring 3.2 will introduce support for PATCH, so @rstoyanchev tells me.
• I am not a big fan of using fragments (#) in URLs. I know that Twitter et al started using fragments for non-standard purposes, but I still think we should stick to the URI RFC:

the optional fragment identifier [...] consists of additional reference information to be interpreted by the user agent after the retrieval action has been successfully completed. As such, it is not part of a URI, but is often used in conjunction with a URI.

In other words: what's the advantage of using a fragment in http://localhost:8080/baseurl/person/1#accounts as opposed to a simple slash: http://localhost:8080/baseurl/person/1/accounts ?

• I am not a fan of having IDs in the representation either. As far as REST is concerned, the whole URL is the identifier, not just the integer that might be at the end of it. Apparently frameworks like ExtJS need this ID, but I wonder if the goal of this project is to make it easy to connect to ExtJS, or to make it easy to expose "proper" RESTful services.

[jbrisbin]

Thanks for the input, Arjen!

w.r.t.:

• Context: should be simple to add what an Entity looks like, though probably less meaningful to show a CrudRepository, as they're all the same. Nonetheless it might not be a bad idea to link to the repository docs.
• $ref: Will do on the linkage.
• PATCH: I did actually consider that and I think it would be great to support it too. I just wasn't sure whether it's too "new" to support right off. So would that imply that excluding a property on PUT is the same as setting it to null?
• Fragments: I like the fragment because it differentiates a full-blown resource from a subset of that resource. In reality, it probably doesn't make that much difference whether that last bit is separated by a slash or a hash or something else. But it seems to me that it indicates that the fragment portion references a subset of the parent resource (which it does) whereas a slash would indicate a full resource (which it isn't). I actually don't agree with the spec here. I think a fragment is a part of the URI that has some significance that wouldn't otherwise be indicated by its absence and the line between UA and server is being purposely blurred. The URI spec is 14 years old now, after all and the demands of applications in 2012 aren't what they were in 1998. :)

If fragments are not desirable (and having the cheek to disagree with the spec is inadmissible ;), then there's other ways to indicate a property of a resource as distinguished from a real resource. I just need to change the way URIs are resolved in order to recognize them. Fragments are super easy to use because the built-in Java URI class understands them and separates them from the path property. That said, I've also thought about putting it in a query parameter:

http://localhost:8080/baseurl/person/1?property=accounts

Which is much closer to what we we're doing, IMO, than by adding a property name on the end (it's just a bunch of "unnecessary" characters). This would be a natural fit given we're supporting querying entities as well:

http://localhost:8080/baseurl/person?name=John+Doe

The former would be like querying a property of an entity, rather than a subset of all entities, as in the latter.

[markpollack]

Just to clarify on the driving use case in the last comment, it is to make the data sitting inside spring data repositories easy to consume from the browser via the popular javascript libraries. There are two levels to this, one just using a standard ajax call in jquery for example to update a html table, and the other is some more dedicated server side support for javascript components, for example table components.

The display of data in the browser being a primary use case sort of drives the decision about why not use PATCH, as I suspect that might not be all that easy or natural to do from javascript libraries. For PATCH I suppose we can have that as an option to turn on and off.

I would be interested to know how far the gap is between making it easy to connect to jquery/extjs vs. a more idealized "proper" RESTful service. Assuming the gap is relatively large, should we offer both and if so, what is the target audience for the "proper" RESTful service?

[jbrisbin]

w.r.t. ExtJS: Yes, I think the goal is to provide easy access to JavaScript toolkits for building RESTful(ish) single-page services.

While it's important to not do violence to the propriety of REST conventions, I think the greater value to developers lies in easily tying into their existing (and future) JS toolkits than it does in creating something that is "proper" but harder to use (which means it won't be).

[poutsma]

With regard to the use of fragments: in a servlet environment, it is pretty much impossible to get the fragment info out of the HttpServletRequest. See this stackoverflow question for instance. I've done some testing on a Tomcat container, and none of the methods on HttpServletRequest seem to have the fragment info. So I guess that rules out using fragments.

With regard to ExtJS: I think it's perfectly fine to target ExtJS and similar frameworks. I just wouldn't call the resulting code a REST api though, but rather a ExtJS Datastore. I don't think you should try and do both in an initial release, things are complicated enough as it is.

[odrotbohm]

Sorry for the email spam, I somehow have to have overseen this ticket. Apologies for that one. I basically agree with Arjen regarding the usage of links, fragments etc. No need to invent something new here, it's all proven to work (e.g. in Atom).

There's a paragraph in the Links section "One could mix…". I think we now agreed two times on exposing relations to linked resources via /${primary}/${id}/${linkedPropertyName}, e.g. /person/1/accounts. No need to mess with two different ways of representing links. This accounts resource can then be posted to to append new accounts to the person with entity id 1:

POST /persons/1/accounts
Content-Type: text/uri-lists

/accounts/23

This would result in:

201 Created
Location: /person/1/accounts/0

We shouldn't expose plain integer ids into representations as this creates the impression you could send those ids when POSTing/PUTting whereas the URI the request goes to actually is the id. So if there's really a reference to "itself" in the representation it should be a link with the well known rel of "self". The /person/1/accounts resource can be DELETEd (removing all links), POSTed to (to link new accounts) and PUT? to (to replace the entire assignment).

Beyond that it's kind of weird that we discuss the assignment approach as too chatty in the first place but want to allow access to individual properties via a request parameter. I'd rather go down the media type route at a much later point in time where a user maybe somehow can configure the properties actually exposed and bind this to a media type, e.g.:

application/vnd+foo -> person.firstname, person.lastname

so that clients can then use this well-defined mediatype to request a certain view on the representation. But once again, I think we're discussing details here before we even got the rough picture. There's absolutely nothing wrong with adding support for certain query parameters for resources later on.

[odrotbohm]

Maybe yet another command regarding the "suitable for ExtJS" VS. "real REST" debate. I'd argue a well defined REST API is perfectly suitable to be consumed by JavaScript clients. What are we doing here exactly? We're defining a media type, a custom media type. Thus no out of the box client in the world will be able to understand our media type, know where the links are to follow etc. If we wanted so, we'd have to stick to Atom, RSS, what have you. Thus you'll inevitable need to create a client dedicated to be able to work with the media type we define. I don't see a reason why coming up with this client should be more complex if the API is really RESTful instead of if it's tied to a particular client library. I don't see value in tying the service exposed to a certain client library as this makes the usage for other client libraries harder and I didn't assume we wanted to expose data services per client library, do we? That's subverting the entire idea of a REST web service IMHO.

[markpollack]

Hi, Just a short reply. There is clearly no set of two people in the world that can agree on what a restful client API is. I propose to in the end call it a webapi that has tried to incorporate 'restful' design principles.

The focus of the effort is to ensure data can be easily consumed by the few 'defacto-standard' JavaScript libraries out there. It is a client centric view and in the end to support 'vendor' specific component libraries we will have vendor specific server side code.

I think one issue here, and I'm to blame, is not setting out a clear enough vision and also not getting all the people together in the same room to do collaborative design. The email / remote thing I think is creating a bit of tension for no good reason.

Chat with you more later - on my way to the office now.

Mark

Sent from my iPhone

On Jan 26, 2012, at 7:52 AM, Oliver Gierke reply@reply.github.com wrote:

Maybe yet another command regarding the "suitable for ExtJS" VS. "real REST" debate. I'd argue a well defined REST API is perfectly suitable to be consumed by JavaScript clients. What are we doing here exactly? We're defining a media type, a custom media type. Thus no out of the box client in the world will be able to understand our media type, know where the links are to follow etc. If we wanted so, we'd have to stick to Atom, RSS, what have you. Thus you'll inevitable need to create a client dedicated to be able to work with the media type we define. I don't see a reason why coming up with this client should be more complex if the API is really RESTful instead of if it's tied to a particular client library. I don't see value in tying the service exposed to a certain client library as this makes the usage for other client libraries harder and I didn't assume we wanted to expose data services per client library, do we? That's subverting the entire idea of a REST web ser vice IMHO.

---

Reply to this email directly or view it on GitHub: https://github.com/SpringSource/spring-data-services/issues/1#issuecomment-3668392

[jbrisbin]

I think the debate about a purist REST approach vs a pragmatic one is necessary but divergent to our purpose.

Insofar as we are able, we hope to stay true to the spirit (if not the letter) of best practices for building REST applications. I think it's unfair to say that any application that doesn't adhere strictly to the arbitrary guidelines laid out by a small number of people (but not put into universal practice, as evidenced by the plethora of ways to accomplish the same thing) is not "RESTful". Personally, I get tired of hearing that argument put out there and whether you change the name to a "web API" or the "Bob API" makes no nevermind to me, as long as people will stop arguing about it so we can move on. :)

In the end, a REST application that suits the purists will not likely be very valuable to a startup developer that wants to quickly mock an application they want to get funding for. So pragmatically, it's absolutely essential to keep in mind the users of common third-party libraries that have very large user bases (like ExtJS) and target them first since they're the most likely to use (and have already expressed interest in) something like this. The popularity of things like CouchApps shows us that people are more interested in questions like "does it work?" and "does it stay out of my way?" than they are in "does it adhere to pure REST principles?". I'm not saying keeping the ideal in mind isn't important, I'm just saying that if the two approaches are in conflict (the purist vs pragmatic approach) then the pragmatic approach should take precedence.

...which in my mind also precludes targeting Atom as a primary representation. It's a great way to demonstrate these concepts (particularly w.r.t. links) but really unfair to use as the basis for an application design because it understands links natively. I've yet to come across a developer that has expressed interest in writing an Ajax application based on Atom. I'm sure that will be useful in enterprise scenarios and places where offline processing can do funky things with the DOM, etc... But since a JavaScript grid or chart can't, so far as I know, be powered by Atom, IMO it makes sense to restrict our examples to only JSON for the time being.

...which means whatever is easiest for the developer should be how we expose links. Simpler is usually better, so the bare minimum (and default) representation is (and always was, in my mind) text/plain (not an unusual and unknown media-type) with a list of links. But if you're writing a JSON-based application, I would not personally appreciate having to switch representations just to handle links. If I'm using JSON in my application, I'll want to stick to JSON everywhere. And since there's no way to represent a "link" in JSON, which has no concept of linked data at all, whatever we do will be essentially arbitrary. That means simpler is usually better, so a bare minimum in my mind is { "$LINK_KEY": "http://..."} where LINK_KEY = some arbitrary key name that is not likely to conflict with property names. Prefixing a $ on it seems to make sense, as that's likely to almost completely eliminate name collisions with bean properties, leaving only a small chance that non-bean resources might potentially clash. Since $ref is used in MongoDB and in the dojo project, it seems as good a way to represent a link as any, and simpler than including a nested object that tries to mimic the structure of a element.

Referencing a linked property directly (meaning the last bit of the URL is the property name) is fine by me, but again we get back to what's easier. The easiest way to support a list of links in an entity coming in from a browser is to support a mixed list of arbitrary objects, where some might be links and some might be actual objects. The presence of the $ref tells me which of these two options it might be, and the metadata about the object I'm trying to populate from the values coming in tells me that I need to save these new objects first. Having this discussion does tell me I need to create some sample HTML pages that include jQuery et al to demonstrate what I'm thinking here. They should be good testing tools as well as discussion tools.

I'm thinking primarily: "what's the easiest way for the user to interact with this resource via jQuery.ajax or using the Sencha Rest helper"? Immediately following that question is: "how do I turn that functionality into a method call"? That's where the abstractions I've got will come into play, though it's not obvious at the moment how that will happen. In my defense, I've been spending my time on design and the proper level and placement of the right abstractions rather than documenting my thought process. Without knowing where I'm headed, it'll be hard to get a handle on why I'm approaching it this way. I hope that, as soon as I have some web components built up around it and you can start sending it curl requests (which I'm working on right now), the method behind the madness will become more evident.

[odrotbohm]

My points where the following ones:

• I don't see where a well defined REST API is in the way of easily creating a client with any JavaScript framework
• I don't see SD services as a endpoint-per-client library interface
• I have not proposed Atom as media type to be supported. I just argued that we're defining a custom media type which inherently means one needs to create a special client anyway. So there will be glue code necessary for every JS library no matter which one it is
• I still fail to see why there should be a tension at all. Design a web API following REST principles and it should be an easy thing to code against it from any client library (JS, iOS whatever)
• regarding the link format: we can disagree on the format they shall use { rel : 'foo', href : 'bar' } VS. { foo : 'bar' }… but that doesn't matter at all in the first place we have to agree on what to expose as links and how the resources and representations shall be structured in general.

[odrotbohm]

I've just reflected all the stuff we discussed yesterday in the Wiki page, added JSON representations to the examples and JSON Path expressions alongside the XPATH ones.

[markpollack]

Just to summarize, if at all possible. The target user is someone making ajax calls from jquery, extjs, dojo. There are perhaps not that many areas where we need to explicitly accommodate our design for the client library, but we no need to keep that end user in mind and in the end test access from those javascript libraries.

An example where such a viewpoint is relevant is with respect to PATCH support. We can support it as an alternative to PUT, but it in all probability wouldn't be as straightforward to access via ExtJS etc. That doesn't mean we don't support it, but it isn't priority and we can decide once it is there in Spring 3.2.

The discussion yesterday was good.

Just tie off the 4 points that Arjen mentioned. The first point more general description of Spring Data Repositories (2nd nature to us but not everyone else) need to be addressed. Also I think some clarification on the terminology section on that page, I'll take a shot at it.

There was a flip in mindset to not use $ref and instead go with the http://json-schema.org/json-ref style. It was left open to have a media type for dojo that would use $ref if that is more natural.

The suggestion for default media type was application/x-springdata+xml,application/x-springdata+json. need to verify that javascript libraries are happy with that.

No fragment usage.

I don't think we got closure on if we want to expose aggregate style functions such as 'count' then as a resource, e.g. /base/person/count.

There are other directions we can go in the future, OData, very specific support for 'vendor' components, e.g. http://datatables.net/ but we will take that up later.