search

PostgREST / postgrest

REST API for any Postgres database
https://postgrest.org
MIT License
23.42k stars 1.03k forks source link

PostgREST sister projects. #402

Closed calebmer closed 8 years ago

calebmer commented 8 years ago

I had an idea in #401 which I want to explore in its own issue.

First the warrant for my proposal:

PostgREST has been exhibiting a lot of growing pains. Auth was the first, then versioning, GraphQL, PostGIS, RAML, and more. There is so much we want to put in PostgREST that doesn't make sense from an engineering standpoint. The question is how can we extend PostgREST with all these features without compromising core?

Here's my idea:

We split some of PostgREST's common functions into a core library and then create sister projects which will listen on their own port, composed by nginx or another proxy. The sister libraries I initially recommend are:

I recommend we release the 3.0 we have at Christmas, but over the next year make this modular version the 4.0.

(Idea, how awesome would it be to compose our modular system with stdio pipes? Example postgrest-auth --schema auth | postgrest-schema --schema public | postgrest-resources --schema public | postgrest --port 3000).

calebmer commented 8 years ago

Sorry, I accidentally hit publish when the OP was unfinished. It's finished now.

alfonsodev commented 8 years ago

+1 for @calebmer proposal. Regarding authentication server here are two projects that could be used as api gateway

They provide some extra features worth to have a look.

ruslantalpa commented 8 years ago

Since the proposal is for "sister projects" i have nothing against it :), i just have a few questions in regard to proposed sister projects. The first 3 (resource, rpc, schema) are currently all in one. What would be the benefit of splitting it in separate projects/binaries (and later combine them with nginx)? The 4th, auth server, any system needs that but my question is what would be the benefit of implementing in in haskell. Some time ago @calebmer mentioned a nodejs lib that provides >100 types of authentication. Why not use that (point ppl to that) to do authentication, it does not seem to me that the auth part gets much "load" in a system so i don't see much benefit from it being implemented in a compiled language.

+1 for things like kong/tyk.io they are interesting and we should study projects like that to see how postgrest fits in.

PS: the unix pipes, i am not sure how that would work. The command line looks nice but the implementation would be a nightmare. The information can not flow back in a pipeline, and it needs to because only the first process in the chain has access to the tcp socket of the client.

begriffs commented 8 years ago

I don't think core is getting too crazy yet. We did a good job with JWT so that other systems can do our auth for us. We already generate docs from inside the core binary, and an extension of those docs probably won't be insane and will also be in its own module. PostGIS support just means adding a few more operators if I understand it right.

The unix philosophy is great of course, although truly stream oriented servers that compose with a shell pipe seems more like an analogy than directly implementable.

If we keep the code modular and clean we should be OK being monolithic for a while. My instinct is to keep the binary in once piece until we really do feel the pain in the codebase.

calebmer commented 8 years ago

I think this is a long term plan if nothing else, but I'll still clarify a few things.

@ruslantalpa the advantage of breaking core down into 3 parts is modularity. As we've been talking about schemas there has been some want for RAML, some want for JSON Schema, and maybe some people who don't even want self documentation at all. I do believe we should also have one core binary which encapsulates some of the main modules we develop which would operate as the binary of today.

To your next point @ruslantalpa, the nice thing about splitting things into modules is we could build a node module. However, I do think we need a native PostgREST auth solution (written in Haskell, takes advantage of Postgres features) which contains industry best practice for people who want PostgREST to be completely usable fast.

@begriffs I agree, I do think this is a long term evolution of PostgREST as it tries to become a staple in the HTTP API field.

And finally, literal UNIX pipes is probably unrealistic, but another similar system might be good if we go this direction.

begriffs commented 8 years ago

Closing this broad issue. Let's discuss the specific parts of this suggestion as their own issues when they become a problem.