Service client

[benblack86]

Second attempt at service client with built in load balancing. Some of this code can be removed (hang over from previous attempt), but I wanted some opinions on whether this is a better approach. I thought harder about how it will be configured, which resulted in the load balancer being more integrated into the service client logic. I'll add comments to the important parts.

@DanSimon @amotamed @aliyakamercan @jlbelmonte @dxuhuang

[benblack86]

In response to your first point, I think we can make it updatable.

I don't think forcing every client to use a load balancer is a problem since in the basic case of one client and no retry policy, we can just call send directly without extra wrapping.

At least for our use cases at Tumblr we hardly ever create a raw client. Where we do use the raw client it is only because we don't care about failure. However, in most cases where we use 3rd party http clients (because we need to use SSL) we tend to have retry logic. Having retry logic built into colossus clients would cut down on this copy & paste retry code.

On the issue of "framework of frameworks" vs "application framework", I'm of the opinion that colossus be used in the pursuit of building (micro)services. Our use cases all point to an application framework. I don't think that is a big change from what we have. I think that means adding more functionality (http compression, SSL, request tracing) and making colossus easy to use (load balancing built in, more documentation). If we can build a framework that handles all use cases at Tumblr, I think we will have created a framework that covers the majority of other people's use cases.

I find it difficult to visualize what a "framework of frameworks" would be or how we get there. What are the use case for other developers? If we built a new application framework on top of colossus, I would be concerned that would slow down development as we would have two frameworks to understand.

[DanSimon]

I don't think forcing every client to use a load balancer is a problem since in the basic case of one client and no retry policy, we can just call send directly without extra wrapping.

I don't think we can do this if we also allow the host list to change. What happens if you start a client with one host, no wrapping, and then want to add a second host (not too concerned about the vise versa)? But if we can come up with a solution to that, then I have fewer qualms with the PR.

The "framework of frameworks" vs "application framework" drove a lot of the decisions in 0.9.0, but I realize I wasn't completely clear about what I meant with that. I agree that at the end of the day, Colossus is a framework to build services. As a user building a service, I want to have as little friction as possible. The whole FoF vs AF (hopefully I don't start introducing too many abbreviations) is primarily to drive how we properly support different notions of what a "service" is and how we organize the code.

0.9.0 had a major shift where protocols now own almost all of the traits/classes that users directly interact with. For example, users no longer use ServiceClient directly, but instead use a protocol-specific interface like HttpClient etc. This is because every protocol works differently and needs to support a different set of features, and doing things this way helps avoid "kitchen sink" designs where we try to build everything possible into monolithic core classes. This doesn't scale, so instead we need to think of each protocol as a user itself of a more generic FoF that provides its features in a modular and composable fashion, such that the protocol is able to build its own types and API with only what it exactly needs.

That said, I do hope we can keep things modular enough so if somebody wanted to use all or just part of Colossus as the basis for a larger framework, they could do so without having to hack around a lot of stuff.

[benblack86]

My plan is to update the client to make it updatable, clean up the code and write docs. Will reopen once ready.