Netflix / staash

A language-agnostic as well as storage-agnostic web interface for storing data into persistent storage systems, the metadata layer abstracts a lot of storage details and the pattern automation APIs take care of automating common data access patterns.
Apache License 2.0
209 stars 45 forks source link

any reason for not using something like OData (with HTTP/JSON) for the REST API? #5

Open jstrachan opened 10 years ago

jstrachan commented 10 years ago

so I've long wanted a nice clean REST API for working with databases; that hides as much as possible the physical storage details of the gazillion different kinds of DB/storage we have these days (SQL v NoSQL v ElasticSearch / Hadoop et al) never mind caching and replicating state to different storage engines etc. I'm surprised we still don't have a good standard thats widely adopted. So good luck on Staash, I wish it well!

App developers usually wanna just read/write blobs of (say) JSON and not worry too much about exactly how a particular database implementation denormalises (or not) or whether its column / key-value / SQL based etc.(e.g. like the ElasticSearch / Mongo APIs)

I wondered; particular since Netflix did host an OData source for its catalog a while back; if you had any particular reasons for making a new REST API or not just reusing OData? I wondered if a little FAQ on the wiki might be handy about why Staash is doing things differently? e.g. whats different or bad with OData; lessons learnt working with OData etc?

From a web app developers perspective breezejs looks pretty neat; I wondered if we could layer it on top of Staash some day? (Though it might be a bit too OData centric for Staash) - at least for the query & insert/update/delete parts it'd be handy.

Though I guess app developers often want a REST orchestration API for working with databases; which is probably a layer of abstraction over databases/tables/rows etc. (e.g. defining more like DTOs than entity beans to use JEE kinda jargon).

Anyway, keep up the good work!

shyamalan commented 10 years ago

Odata seems to be more focussed on navigating connected entities and is oriented towards being meta-data driven , from the general use-case point of view it would seem like a bit of an overhead for some of the scenarios we had in mind, though it does provide a higher level abstraction and is a great fit for certain situations, eventually the application will be coupled with odata service via the metadata and links in the responses. Thanks for the pointers , I will take a look at those in more detail.

sblom commented 8 years ago

OData's greatest strength is that it enables fairly rich ad hoc queries over your entire dataset. OData's greatest weakness is that it enables fairly rich ad hoc queries over your entire dataset. For internal applications, OData is usually quite quick to set up and there are some useful tools for doing ad hoc exploration of the dataset.

For public applications, I would caution against shipping an OData endpoint. From personal experience, the right way to think about an OData endpoint is as a SQL connection string over HTTP. Once you ship an OData endpoint, your users will be free to query/filter/sort over your data in any way that they choose from now until the end of time. If you ever discover that particular use cases would benefit from optimization, good luck because you'll have to write code to spot specific query patterns and route them to a different processor than the generic OData one.

approachings commented 3 years ago

some years later, as OData and the internet have changed; I'd say the above mentioned reasons no longer apply

if you need an optimization endpoint, you can just make an new entity type for browsing the optimized results

as far as publishing, you can have pretty good security going on with OAuth now so if you really need, just have expiring authentication tokens and access tokens

OData is a good API as long as your API is geared toward sharing data. if the main focus is triggering actions through web services, OData is not well suited for that. for example transactional, multi-step APIs in which you generate/send/create entities the API will operate on

for example mass geocoding could be done through OData but it would not be very natural to shoehorn bulk data input in