Allow projections by listing properties to be returned [DATAREST-264]

[spring-projects-issues]

Sri opened DATAREST-264 and commented

Currently accessing a resource will list all properties of a resource. However in certain cases, clients would be interested in certain properties of a resource. Of course, client would know about available properties of a resource through 'resouce/schema' feature of SDR. One way SDR could be enhanced to spit out only requested properties through parameter, say fields.

Ex. /resource?fields=prop1&fields=prop2.... should only JSON with prop1, prop2 ...

This could be applied for resource listing too.

Not sure if my request contradicts REST principles in anyway, but this feature is very helpful to our use case

---

Issue Links:

• DATAREST-1180 Allow runtime projection definition ("is duplicated by")
• DATAREST-221 Support use case-driven inlining of managed resources

5 votes, 8 watchers

[spring-projects-issues]

Oliver Drotbohm commented

You might wanna have a look at what we already implemented for DATAREST-221. It allows to define a named projection through an interface and the projection being exposed as URI template parameter

[spring-projects-issues]

Sri commented

@Oliver DATAREST-221 is different from my request above. In essense my request to hide or show properties of resource is rather dynamic. I also commented the same in above mentioned ticket.

Any progress on this ticket, if I may ask?

[spring-projects-issues]

Sri commented

FYI Google implements this feature something like this

 GET http://www.google.com/calendar/feeds/zachpm@google.com/private/full?fields=entry(title,gd:when) 
```. 
More details [here](http://googlecode.blogspot.com/2010/03/making-apis-faster-introducing-partial.html) and [here](https://developers.google.com/youtube/2.0/developers_guide_protocol_partial)

[spring-projects-issues]

Ruslan Stelmachenko commented

I think dynamic hide or show the properties is most useful in public apis. You never know for what user of your API read the entity. Even in private apis you often need slightly different data.

For example: on one page only item.name (with _link.self) to display items in combobox and on another page item.name and item.address to display them in another combobox. If you will create a projection for every use case, than it will be hundreds of unmaintainable projections in the code. Especially if you will create subprojections for return values of this projections to inline only needed fields of subentities.

But if you can write just: /items/?include=name,description,address.street or /items/?include=name,description,address(street,house,flat) or /items/?exclude=someBigRelation it will be perfect.

I even think that combining projections and on-field-basis filtering is even better. We can declare our base projections and slightly change them on-the-fly for client's needs.

About problem with PUT. This problem exists in projections-solution too. I don't know how to clearly solve it. Maybe produce an error if user PUT too low data. But what to do if user PUTs too many (embedded entities)? Maybe do PATCH them if their IDs doesn't changed and if changed then do relations change as for text-url requests. Maybe just ignore changes in embedded entities. In the end we do PUT to items resource. Not to their subresource. Maybe there is a smarter approach. I have too little of expririence in saving data from REST

[spring-projects-issues]

Ben Madore commented

Not sure if it's useful, but I think this library implements the general concept that the submitter is referrering to: https://github.com/skyscreamer/yoga

[spring-projects-issues]

Nico commented

+10

[spring-projects-issues]

Sam Kruglov commented

I think if this feature gets released it would most certainly not be "minor". Please, consider increasing priority

[spring-projects-issues]

Oliver Drotbohm commented

There are no plans to actually add that for a multitude of reasons:

A feature like this moves much fine-grained control to the clients whereas in REST that lives on the server. SD REST is not a data access API via HTTP & JSON. It's a foundation to build hypermedia-driven, high level REST services. For more details on this read my comment on this InfoQ discussion thread.

The requested feature is shifting the behavior of the exposed resource in the direction of a select clause of a database query. That's not an abstraction level Spring Data REST operates on, as we basically take what the repositories expose and translate that into well known HTTP resource implementation patterns. I.e. there is simply no way for us to properly hand this information down into all repository implementations. It pulls low-level functionality up to a level where a high-level business interaction should occur. That's wrong. Those decisions belong into the repository layer, not in front of it. Because otherwise you're back to the abstraction level of a low-level database access API. I fail to see why you'd want to operate on such low-level on a high level protocol like HTTP as you basically get none of the benefits (evolvability, cacheability (see below)) and at the same time accumulate all of the downsides (e.g strong coupling) and pay for all the extra effort a protocol like HTTP comes with. For reference, see Stefan Tilkov's slide deck here (there's probably also a recoding somewhere). SDR wants to live on the upper edge of the "Domain logic" layer, so that users can focus on building the "Process flow" part. What you are asking for is moving down to "JDBC in disguise". That's simply not the goal of the project.

An API that works like this fundamentally undermines the cacheability of resources as you create a very fine grained set of resources, so that you actually might see worse performance (of your server) with a very detailed query as it has to hit the application and the database instead of being answered from an HTTP cache, just because you wanted to save those 20 extra bytes in the JSON response. With payload compression enabled those couple of extra bytes hardly matter on the clients in the first place, compared to the addition complexity and effects they have on the rest of the infrastructure. If a client really benefits from a dedicatedly shaped resource, by all means go ahead and design and implement one

[spring-projects-issues]

Sam Kruglov commented

Thanks, Oliver! That was a very useful read. I agree with that SDR works on presentation level and should not bother with constructing database queries. However, I still have 2 concerns:

worse performance (of your server) with a very detailed query as it has to hit the application and the database instead of being answered from an HTTP cache

1. I am not an expert, but I doubt that SDR actually caches data from the database. An incoming REST request still must go through JPA layer in order to at least construct the JPQL and then SQL and so on. So we still must hit the application layer anyway. Also, I have not yet worked with caching in Spring Data but I believe it does not just work out of the box and we must use annotations like @EnableCaching, @Cacheable, and @CacheEvict. —
2. I have only just seen that we're talking in Spring Data REST context for some reason. I think it is supposed to be at Spring Data level, like Spring Data JPA. I think that could be a useful feature. Although IMHO using @JsonFilter is enough for such cases even though it does not affect SQL.

— A little off-topic: where do you think is the right place for a custom non-RESTful (or/and non CRUD) API? Should it be within /search path and be linked from /profile or should it reside on a separate basepath? In my case the whole API is internal and we just use SDR for convenience

[spring-projects-issues]

Marcel Overdijk commented

If you want to define which properties to return per request, then graphql might be interesting to look at

[spring-projects-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.