API gateway example - Githubissues

peterbourgon commented 8 years ago

A nice example might be an API gateway, which hosts routes like /api/:service/:method, and demuxes them to individual (defined) services.

justintv90 commented 8 years ago

Interesting. Cant wait for it

ghost commented 8 years ago

A gateway has to provide security since its the layer facing the outside world.

ghost commented 8 years ago

Would nginx be a good fit for this function?

peterbourgon commented 8 years ago

Go's net/http is suitable for use on the public internet.

michaelbironneau commented 8 years ago

We were thinking of making an API gateway too. Found out about go kit at Golang UK today and it looks like the perfect set of building blocks. My thoughts were something like this:

As you said, routing, but maybe more dynamic. The gateway should let services register/deregister, specifying a mount point relative to the API root. The registration process would involve passing along a machine-readable API specification, eg. Swagger that the gateway uses to extract routing information.
An endpoint exposing a single machine-readable documentation (merged over all services). This is unlike existing API gateways that have one documentation per service and don't attempt to merge them, at the expense of the end user who must then reason about all the individual microservices' APIs instead of just one API.
Pre/post request hooks to allow request transformation (or additional logging/instrumentation), either via adapters in Go (in which case it needs to be recompiled) or via external HTTP services.

Please feel free to disagree or suggest something entirely different :). Also, some of this stuff is probably out of scope for just an example, but it could certainly be built up from said example...

peterbourgon commented 8 years ago

Thanks @michaelbironneau! This gives me a good idea for what an API gateway example could be driving towards. I think some of the more dynamic bits would indeed be out of scope, but it's important to understand them to build a good foundation.

tmosleyIII commented 8 years ago

I think a good example to look at would be Tyk API Gateway, it's also written in Go.

ghost commented 8 years ago

Caching by service could be invoked in this layer so caching could implement specific service requirements. Responses could be selectivity stored as a data source for mocks. The whole issue of service testing needs to be addressed.

zakkwylde commented 8 years ago

Hi,
any update on when this example might be available?

Thanks

peterbourgon commented 8 years ago

No, I'm sorry. It's in the queue :)

rmuhzin commented 8 years ago

Hi @peterbourgon, any updates on this?

Thanks for the amazing toolkit :+1:

peterbourgon commented 8 years ago

Yes, I've done some thinking, and spoken to several people online and at conferences on the subject. I'm now reasonably convinced that the API gateway pattern is actually an anti-pattern. I'd be very curious to hear the arguments otherwise, if anyone in or subscribed to this thread would be willing to make them...

ghost commented 8 years ago

Depends on what you mean by API gateway. Please explain.

michaelbironneau commented 8 years ago

Just to be clear, for me an API gateway means "a single point of integration for clients of the system", where the system in this case is composed of lots of little (micro) services, some that may be exposed to clients and some not. But the point is that the client does not see that - they just see a single service.

If the alternative is to make clients deal with every service individually that has got some issues of its own IMO (though probably some upsides too). For example, if one day you decide to change your service boundaries that becomes impossible without major breaking changes for clients. If you were using an API gateway that could be entirely transparent to the user.

I'd love to know what downsides convinced you that API gateways are a bad idea in general?

peterbourgon commented 8 years ago

For me an API gateway is a single service that hosts endpoints that effectively proxy to other services. I guess a lot of orgs use nginx or Haproxy as a defacto API gateway. This is fine and makes a lot of sense.

But an important additional property of API gateways as I understand them is that they dynamically figure out what other services exist at runtime, likely by consulting some system of record (e.g. Consul). They may also read some kind of schema, to figure out what routes (endpoints) are available on each, and maybe perform some kind of validation. It's this that I object to. It introduces nondeterminism and spooky action at a distance into your infrastructure. In the boxes-and-lines drawing of your architecture, every line and every box should be carefully thought out and explicitly placed. This is what Go kit encourages you to do in the small, when wiring up the component graph of your service in your func main. It is also what Go kit should encourage you to do in the large, by ensuring all your service-to-service communication is explicitly enumerated and specified.

If we're just talking the first scenario, where all services are explicitly configured and can't change during runtime, then I don't have any problem with this, but I also don't think it makes for a very compelling example. The code would be very straightforward. But the second scenario is what I'm objecting to. (And I'm happy to hear arguments otherwise!)

basvanbeek commented 8 years ago

I've been thinking about this as at first I was a supporter of the idea to see an API gateway example but looking at the current goals of Go-kit, what it tries to solve and which components it consists of, it really doesn't make a lot of sense to provide such an example at this point in time. Currently all parts deal with the micro services themselves and how they can properly be called, exchange data through RPC, handle misbehavior, log items, etc. Basically everything internal to the platform you're building. A gateway implies talking to the outside world: apps, websites, external clients, etc. Go-kit does not have any opinion or code dealing with authentication, authorization, CORS, sessions/tokens, whitelisting/blacklisting, etc. Nor does it make a choice on a mux (net/http, gorilla/mux, HttpRouter, Goji, etc.)

Building an example leaving these items out would not be much of an example and does not benefit prospective users of Go-kit. If Go-kit would implement or have an opinion on the packages to use for these items it might make sense but for now I'd rather see us focus on improving the existing code base and adding new features and integrations to handle the internals.

peterbourgon commented 8 years ago

@basvanbeek I agree with everything you've written. Well-said.

anthdm commented 8 years ago

@basvanbeek :+1:

roylee0704 commented 8 years ago

Based on the conversation,@peterbourgon mentioned that API gateway is anti-pattern to Go-kit. Does that mean we are not recommended to build API gateway with Go-kit?

I'm sorry if this question sounds trivial, and I have no intention to reopen the issue, I'm a beginner trying to understand how/why :)

peterbourgon commented 8 years ago

Yes, my thinking has evolved. I filed #202. Sorry for my apparent flip-flopping :)

go-kit / kit

API gateway example #91