search

openservicebrokerapi / servicebroker

Open Service Broker API Specification
https://openservicebrokerapi.org/
Apache License 2.0
1.19k stars 433 forks source link

Add support for backup & restore to the service broker api #107

Closed avade closed 7 years ago

avade commented 7 years ago

Problem

As a service user I want to have confidence that my service instance is backed up and know that if something bad happens that I can restore from a backup.

As a service operator I want to offer service users a self-service mechanism to perform backup and restores of the service instance, so I don't have to manage this process.

Workflows

As a service user I can trigger a backup for a given service instance

As a service user I can list all backups for a given service instance

As a service user I can restore to an existing service instance

As a service operator I can choose not to offer self-service backups

As a service operator I can choose not to offer self-service restores

Next steps

  1. Community feedback on this feature
  2. Create proposal google document
duglin commented 7 years ago

No strong opinion on this one yet but wanted to add that I thought these types of operations are normally considered to be extensions to the SB spec. Meaning, a service provider may offer these to their users but they would be too specific to include as part of the core API. So, for example, they may be available via the admin/console URL provided by the service. But definitely something we should discuss.

avade commented 7 years ago

I think backup & restore is supported by the majority of services platforms in a generic way for traditional data services [1].

The improvement to the UX for developers vs going to a dashboard that's different for each service / service provider for a common operation like backup / restore is worth extending the API for.

Agree this is a good discussion item for the next call.

[1] Compose.io is a great example: https://help.compose.com/docs/compose-backups

duglin commented 7 years ago

Just need to be careful in assuming that all services are "data services". I have no idea what % fall into the "data service" category and which don't but I do know that its not 100%.

avade commented 7 years ago

Good point, I would imagine this feature would be similar to bindable in the API today. This is an opt-in feature and brokers that don't support binding do not need to implement the endpoint.

angarg12 commented 7 years ago

@avade does this assume that SB is the one delivering the service?

In our use case SB is a gateway to different service providers, so effectively the only state stored in the broker is the instances and bindings information.

In our system this would be implemented via "events". We expose an API where users can send events to service instances. Backup is a perfect example of such an event.

That solution works for us because different providers offer different events (i.e. not all services provide backup) so we keep the API consistent all across the board.

gberche-orange commented 7 years ago

Community feedback on this feature

I have no idea what % fall into the "data service" category and which don't but I do know that its not 100%.

Support for additional life cycle events beyond the current CRUD (Create Read(Bind) Update, Delete) is definitively needed for different use-cases, including:

An alternative to a normalized backup verb in the SB API, is the notion of service broker action proposed by @benlaplanche . The SB declares a set of additional actions supported (e.g. in a similar way than #59 with a JSON schema) that platform UIs (e.g. CF CLI, or Web UIs) can discover and use to expose extended actions in a uniform ways to platform users.

In our system this would be implemented via "events". We expose an API where users can send events to service instances. Backup is a perfect example of such an event.

@angarg12 The events concept seems interesting to detail, especially if this could be a concept also to also support other use-cases such as broker to app notification events: credentials renewal/rotation, usage quota reaching limit....

Could you please share related spec URL ? How do you envision adapting this to the SB API ? How would this translate into UX, ie how does this gets leveraged in UIs (CLIs and web UI) ?

avade commented 7 years ago

@gberche-orange The broker actions concept is interesting, but it does mean that we would have to rely on standards for broker action types. For example, broker A calls backups action take-backup broker B calls backups action trigger-snapshot, which is confusing for developers. A schema with descriptions would go some way to give clarity to users.

As always the more generic the spec the more decisions we push onto consumers and broker authors. Adding backup & restore as a first class concept would be taking an opinion that it's something the majority of services would want to support with a standard UX for consumers.

@angarg12 how do service users submit events to the broker via CF? As @gberche-orange said, a spec / example that we could see would be helpful.

paulcwarren commented 7 years ago

Alex, this would be useful for persistence too, I think. A developer with an app reading/writing to an NFS volume may very well want to back up that NFS volume, for example. And later on restore it.

gberche-orange commented 7 years ago

@avade

The broker actions concept is interesting, but it does mean that we would have to rely on standards for broker action types.

The same holds for catalog normalization, where CF community-driven-standards appeared to be both flexible and lightweight without putting that much load on service authors.

Adding backup & restore as a first class concept would be taking an opinion that it's something the majority of services would want to support with a standard UX for consumers.

We would need to detail the variations of backups that are required to support the different use-cases:

In our experience with db-dumper-service and its cli-plugin two "backup" and "restore" verbs are too limited: we need verbs to list/delete/download backups. We also plan to introduce additional verbs to manage backup stores (a container for backups with their quotas and eviction policies...).

I believe that it would be more useful to support additional generic verbs, than try to normalize the many backup/restore variations which only benefits/applies to data services as @duglin reminded, and would leave lots of other services (e.g. route services such as api gateways) without the needed extended verbs they need.

angarg12 commented 7 years ago

@gberche-orange @avade for the record I have asked my colleagues to provide me with the spec/examples on how these actions are implemented in our system. I am waiting for a reply, I will post details once I get them.

avade commented 7 years ago

@gberche-orange good points. I am starting to agree that broker actions may be the way to go. It keeps the API simple but extensible for advanced use cases. We can offer a "community standard" guide as part of the spec, similar to CF metadata.

How about we come up with a 2-4 page proposal for broker actions and reference some of the use cases in the document from @benlaplanche [1]. I have started a new issue for this here [2].

We can then discuss on the call next week if you can join?

[1] https://docs.google.com/document/d/1Y5vwWjvaUIwHI76XU63cAS8xEOJvN69-cNoCQRqLPqU/edit#heading=h.kzfeyua6md59 [2] https://github.com/openservicebrokerapi/servicebroker/issues/114

paulcwarren commented 7 years ago

+1 for broker actions also. We have this story on our backlog that sounds very similar.

gberche-orange commented 7 years ago

thanks @avade sounds great. Proceeding conversation in #114

mattmcneeney commented 7 years ago

@avade Do you want to close this one now that it's being tracked in #114?

avade commented 7 years ago

Yep, added a link in #114 to this thread.