Service Instance Generic Extensions/Actions (Take 2)

Samze commented 5 years ago

Why

The specification defines endpoints that allow the lifecycle management of service instances and service bindings. However, a common complaint is that it does not support some of those important “day 2” operations that developers might want, e.g backup and restore. Also that it does not allow service specific operations, e.g. MySQL set leader. To accomplish this Service Brokers authors have the option to either go off-spec or to misuse the spec (e.g. cf update-service -p ‘{“trigger-backup”: true}’).

The specification needs an extension mechanism to allow authors to define new endpoints.

There was a previous PR attempt to provide this functionality through “generic extensions” which was opened in 26 Jan 2018. This PR proved complex to implement in part extensions could be hosted anywhere and it allowed any authentication via OpenAPI security schemas.

What

This is our first attempt to revive generic extensions!

This PR is similar to the previous PR in that:

It provides extensions that act on a service instance.
It uses OpenAPI to describe the extension.
It has an URN for each extension. The hope is services share extension specs and common tooling can be created.

It differs in that:

The extensions are discoverable from the catalog - allowing users to know what extensions are available before provisioning an instance.
All extensions must be hosted on the service broker itself (although the service broker may act as a proxy if it likes). Since platforms already talk to brokers this simplifies things, e.g. networking issues.
All extensions must have the same authentication method already used between platform and broker. This makes it easier to implement both platforms and broker.

In order to not limit platform implementations we have not provided guidance on HOW the extension is triggered. Since the endpoint is hosted by the broker it is likely that platform makes a request to trigger the extension on the user’s behalf. This is the same as how all endpoints are called today from known platforms.

Open questions:

Should the OpenAPI document be required? OpenAPI allows clients to discover how to consume the endpoints. There are tools that allow for auto generated clients/UX. However it is unknown whether this would be used. Will the people who write tools for the extensions just know how the extension works?
Is it confusing that the OpenAPI paths live on a base route?
Should we be more opinionated about how the extension gets trigger?
Should extensions description be optional?
Do we need to be explicit about how brokers do async operations using this model?

Notes

We have not updated the OSBAPI OpenAPI yet for this PR since we wanted to get feedback first.

This PR came from this doc.

Sam & @georgi-lozev

(If this is unmerged in 2025, please close)

Closes #114

cfdreddbot commented 5 years ago

:white_check_mark: Hey Samze! The commit authors and yourself have already signed the CLA.

fmui commented 5 years ago

Why is the path part of the ID? I think the extension ID should be something globally unique to distinguish - let's say - two different backup and restore specifications.

Here are a few opinions:

The OpenAPI URL should be strongly recommended, but not required.
Because the broker credentials are used for authentication, only the platform can call extensions. I don't see reason to call that out explicitly.
The description should also be recommended but can be optional if the extension ID is globally unique.
The async behaviour depends on the extension. We shouldn't force a specific async flow.

mattmcneeney commented 5 years ago

Two outstanding things for this PR:

How should we deal with the identifier for the extension? @fmui please could you add your thoughts?
MUST service authors define an OpenAPI document for an extension or can this happen out-of-band? Do you have use cases for why this shouldn't be required @fmui - is the intention to keep things as simple as possible? How would a platform know what HTTP request to make?

fmui commented 5 years ago

I would expect the extension ID to be globally unique and not having the extension base path in it. For example, if I see the extension urn:osbext:pivotal.io:mysqlbackups:v1, I know that the broker implements Pivotals MySQL backup. If that is a well-known extension, the platform might not even need the OpenAPI document for this extension (although I would strongly recommend always providing an OpenAPI document). The extension base path would then need to go into another field. For example:

{
  "id": "urn:osbext:pivotal.io:mysqlbackups:v1",
  "path": "/p-backup",
  "openapi_url": "https://pivotal.io/osb-extensions/backup_restore_v1.yaml"
}

If the broker also supports another backup extension, it is free to set a different path for that extension.

Samze commented 5 years ago

I would expect the extension ID to be globally unique and not having the extension base path in it. For example, if I see the extension urn:osbext:pivotal.io:mysqlbackups:v1, I know that the broker implements Pivotals MySQL backup. If that is a well-known extension, the platform might not even need the OpenAPI document for this extension (although I would strongly recommend always providing an OpenAPI document). The extension base path would then need to go into another field. For example:
{
  "id": "urn:osbext:pivotal.io:mysqlbackups:v1",
  "path": "/p-backup",
  "openapi_url": "https://pivotal.io/osb-extensions/backup_restore_v1.yaml"
}
If the broker also supports another backup extension, it is free to set a different path for that extension.

The current proposal specifies the extension path implicitly from the openapi urn. `:extension_path MUST be the namespace specific string(NSS) part of the extension URN plus the path in the OpenAPI document.`. As you suggest, the other approach is that the broker author can explicitly declare which path to use, perhaps that would be less confusing.

Samze commented 5 years ago

We discussed this on the call and decided:

We will have the broker author specify the path explicitly rather than it be implicitly inferred from the id.
The OpenAPI document should be required. If a broker really doesn't care they can always just provide a dummy YAML doc.

I will update the PR.

Samze commented 5 years ago

This has been updated.

mattmcneeney commented 5 years ago

Some folks from Heroku are playing around with an experiment for this right now. They may leave comments in this PR when they have them.

If they have something worthy of presenting they will bring to the group for review. Thanks!

mackwong commented 4 years ago

any update?

linux-foundation-easycla[bot] commented 4 years ago

The committers are authorized under a signed CLA.

:white_check_mark: Sam Gunaratne (d55069c4afd3e18f848cdc6754d3fbd56b8b79da)

Samze commented 4 years ago

I think it need to define an operation as the implementation of extension, just like instance is the implementation of plan.

Could you elaborate on this?

WalkerGriggs commented 3 years ago

I may be reviving a necro-thread here, but...

I mostly wonder why we're opting to couple extensions to service instances? More to the point: can we generalize this proposal further so that the broker itself can be extended?

I'm thinking about additional broker authz/n implementations, for example.

This wouldn't mean a loss in functionality from the current proposal either. In theory, a broker could still specify an instance_id parameter in the extension's OpenAPI spec, but dropping the broker extension path from /v2/service_instances/:instance_id/extensions/broker-extension-path/backup to /v2/extensions/broker-extension-path/backup might give brokers a bit more flexibility.

Thoughts?

pivotal-marcela-campo commented 2 years ago

@Samze Can you provide some insights here or hand off to someone else?

Samze commented 2 years ago

@WalkerGriggs Yeah there are definitely use-cases for both extensions on brokers and on instances. The original idea was to break down the problem into two, start with extensions on instances and then follow up with extensions on the service broker itself. We started on extensions for instances as we had the most practical usecases for that. e.g. backup an instance.

@pivotal-marcela-campo yeah happy to chat more about this proposal if someone wants to take over driving this. It really stalled because while we had agreement in the PMC about this approach, neither CF or Service catalog had done any experimentation to validate this approach.

openservicebrokerapi / servicebroker