Meta: Create two service brokers

[krancour]

This meta-ticket, realistically is not resolved through a PR to this repo, but hey, we gotta track this work somewhere...

Create (or identify) two reasonable, fully functional implementations of a service broker. Both should implement the CloudFoundry service broker API, since CF mode is the only mode currently supported by Steward.

The goal is to demonstrate two very different, but equally plausible use cases:

• [x] A broker that provides "disposable" PostgreSQL databases.
  • Adequate for use in a non-production, self-serve environment.
  • Exposed plans are very generic-- e.g. "small," "medium," "large" (or even just a single plan like "standard").
  • A database (or entire RDBMS; dealer's choice) is created when "provisioned."
  • An existing database (or entire RDBMS) is destroyed when "deprovisioned."
  • Binding provisions new credentials for use by the whomever has bound.
  • Unbinding destroys those credentials.
• [ ] A broker that governs access to a single, long-lived PostgreSQL database.
  • Suitable for production.
  • Exposed plans are very specific-- e.g. "zipcodes".
  • Broker provisions database at startup (if the DB doesn't already exist). Dealer's choice whether the broker also provisions the RDBMS, if necessary, or merely uses an existing one (configurable).
  • Broker does not deprovision database (or RDBMS) at shutdown. (In production, you don't want to lose all your data just because a broker died.)
  • The "provision" action is a no-op. (The shared DB already exists.)
  • The "deprovision" action is a no-op. (Consumers don't get to destroy this shared resource.)
  • Binding provisions new credentials for use by the whomever has bound.
  • Unbinding destroys those credentials.

A final, overarching goal is that these brokers should be useful in presenting a compelling demo of Steward.

[krancour]

@slack @arschles, I spent a few hours this weekend trying to identify existing CF service brokers that fit the criteria outlined above. I was easily able to find multiple implementations of brokers that provision entire RDBMS and/or provision new DBs within an existing DBMS. (Seems quite a popular approach, actually.) This one is my favorite, and the one I'm using to justify checking off the first checkbox in the OP.

What I haven't been able to find are any examples of brokers (for Postgres or ay other DB engine) that no-op provisioning entirely and merely create credentials for accessing an existing, shared DB instance upon bind.

This is not to say that creating such a service broker isn't tenable. I'm certain I can do it quite easily. What I'm more concerned about is that the lack of any existing brokers that behave in this fashion would seem to indicate that others are not viewing service brokers as a means of governing access to existing, pre-provisioned instances or third party, shared resources.

That doesn't mean our notion of doing things that way is wrong. Maybe we're actually super forward thinking. (Truthfully, after much though, I really, really like the idea.) It does, however, pose some very real difficulties in terms of:

1. Broadening operators existing notion of what a service broker does.
2. As noted, there isn't a vast selection of existing brokers that behave this way-- which means we've got to come up with them on our own, or else cluster operators who are early adopters are going to be stuck implementing their own custom service brokers to govern access to service instances.

Based on the above, I would respectfully ask that we, perhaps, give some further consideration to the following ideas:

1. Service instances don't appear in the service plan catalog.
2. Third-party owners of shared services (e.g. cluster operators) pre-provision services using plans available from the service plan catalog. (No custom broker needed.)
3. Provisioned services are published to a separate catalog of service instances.
4. Multiple applications can bind by reference to a single service instance.

[arschles]

@krancour thanks for the research. I gave our claims architecture some thought over the weekend, and now we can talk about it in the context of real-world brokers. I'll address your points in this post

Broadening operators existing notion of what a service broker does

The current claims mechanism does not prescribe a specific broker design, as this point seems to indicate. The architecture is flexible enough to adapt to either type of broker equally. It's equally straightforward to accommodate one type of broker over the other. Note that documentation in this repo is lacking on how to do either is lacking now. I'll make it a priority to enhance it after I finish with some high-level features and testing.

Service instances don't appear in the service plan catalog

They already don't. Only services and plans appear in the service plan catalog.

Third-party owners of shared services (e.g. cluster operators) pre-provision services using plans available from the service plan catalog. (No custom broker needed.)

This operation is accomplishable by submitting a single claim to provision (action: provision). Doing so will tell the backing broker to create the new shared service. An operator can later bind to that provisioned service instance, and use that shared service. Note that I've tested this approach using a backing github sample broker. This broker creates a new Github repository on every provision operation. Binding gives multiple consumers access to that repository.

Provisioned services are published to a separate catalog of service instances

I don't see the benefit of a formal catalog of provisioned service instances. Currently, an operator who provisions a service does so with a ServicePlanClaim. That claim does not disappear, so that's the record of the provision operation.

Multiple applications can bind by reference to a single service instance

Currently accomplishable with ServicePlanClaims.

We're discussing how to represent a state machine within Kubernetes resources. Below is a list of the states:

• provisioning
• provisioned
• binding
• bound
• unbinding
• unbound
• deprovisioning
• deprovisioned

The state transitions create a DAG, and a consumer should be able to enter the machine at provisioning, binding, unbinding and deprovisioning.

Both architectures give access to each each state entry point, state and transitions between each. They can each support both kinds of brokers discussed herein.

Steward should represent these entry points, all states, and all transitions in the most operator-friendly architecture. I prefer the ServicePlanClaim one. I believe it's a more compact, flexible, and easily documentable representation, which makes it more operator friendly.

[krancour]

Broadening operators existing notion of what a service broker does

The current claims mechanism does not prescribe a specific broker design...

Understood, however, even if we intend to support a variety of different service broker types other than CF service brokers, what remains true is that it was CF that initially introduced "service broker" into our lexicon and redefining that term more broadly than they already have might really confuse people who are already familiar.

Service instances don't appear in the service plan catalog

They already don't. Only services and plans appear in the service plan catalog.

They don't at the moment, but the vision that @slack was articulating to us the other day was one wherein owners of a shared, third-party service (e.g. a zipcode database) would make access to that "service instance," available as a plan via a broker-- a broker that isn't generically offering you a "postgres.small" or "postgres.large" or some such, but is brokering access to a specific, existing DB instance. This means things like "postgres.zipcode" would show up in the plan catalog.

The remainder of any of my feedback or yours is probably moot until we establish that the previous paragraph is adequately reflecting what @slack was suggesting and whether:

1. That it broadens the definition of "service broker" beyond how it's generally understood by people already familiar with the term.
2. That it necessitates a different class of service brokers that don't exist-- thereby creating more work either for us or for our early adopters.

[slack]

That it broadens the definition of "service broker" beyond how it's generally understood by people already familiar with the term.

I don't think we are stretching the definition. In the most generic form possible: service broker is the way to expose services to consumers.

The "exposed services" can be anything: access to a zipcode database, a software VPN, a dedicated RDBMS, or a database/username/password on a shared MySQL server. Services all the way down.

The thing that I really like about the ServicePlanClaim approach, is that the way that those services are provisioned and attached is hidden behind the ServicePlanClaim. Operators choose what goes in the software catalog, consumers pick and choose from the menu.

This drastically simplifies the way the consumer operates and places control where it needs to be. The operator publishing things to the catalog.

The other model means you have service instances that become shared resources through the appearance of a reference. That feels like a huge side-effect that is no bueno. The act of making something a shared resource should be intentional, and for me, that looks a whole lot like publishing production.zipcodes in a catalog.

The goal for us is to reduce the burden of creating custom brokers! For example, steward in custom mode should invoke a docker command in a known environment (Steward Container API) for each of the service lifecycle events. That way you can deploy your arbitrarily complex broker in a simple and seamless way.

This reduced friction should lead to an increase in available brokers, built and supported by the community.

[arschles]

@krancour is this ok to close?

Looks like you haven't checked off the second box (A broker that governs access to a single, long-lived PostgreSQL database), but I know we discussed such a broker.

[krancour]

It's safe to close this. I think all of us have a more evolved understanding of what we're after now that obviates the second checkbox in the OP.

(I think that evolved understanding is that although broker can do whatever, our means of managing multi-tenant services to which multiple consumers can bind is not achieved by pushing that onto a broker. It's achieved by Steward taking on the burden of managing the lifecycle for dedicated vs multi-tenant services. See #212, #213, #216)