Consolidated issue for subscription discussion

shilpa-padgaonkar commented 4 months ago

Problem description As discussed here - https://github.com/camaraproject/Commonalities/issues/140#issuecomment-1946047374, I am creating a consolidated issue for the current (explicit) subscription topic. I have broken this topic already into smaller (possible) issues and can create separate issues if we agree to do so. This issue is to to give an overview and please feel free to add comments to extend the scope if needed:

Lifecycle management of subscriptions using common subscriptions.yaml file (issue 1)

Do we need API subproject(domain) specific distinct ways of creating, updating, deleting and retrieving subscriptions? Designing the subscription lifecycle management individually in every API sub-project would lead to the subprojects reinventing the same wheel of subscription lifecycle management each time. They should instead be able to focus on domain specific events. We also need a consistent way for event consumers to subscribe to events across domains.

The subscription lifecycle management related events (SUBSCRIPTION_REQUESTED, SUBSCRIPTION_CREATED, SUBSCRIPTION_UPDATED, SUBSCRIPTION_DELETED, SUBSCRIPTION_ENDED) can also be handled in a common way. Another big change would be that domain specific information cannot be added to subscriptionDetail as is probably done today.

This issue was raised earlier in Commonalities and we decided to park (point4) for later at that time https://github.com/camaraproject/Commonalities/issues/44#issuecomment-1715855946

If we would agree that we need a common way of handling subscription life cycle management, there could be the following 2 options:

Subscriptions.yaml in maintained in a separate/commonalities repo and subprojects simply maintain the event types - or
Subprojects create a copy of the subscriptions.yaml file (maintained in a separate/commonalities repo) from the latest release and just extend it with the domain specific sections in their subprojects
1. scopes
2. security schemes
3. filters
4. configs

One subscription for multiple event types (issue 2)

Camara could define a standard way for subscription managers to allow event consumers to subscribe to multiple event types using subscription/s, instead of making the event consumer create a new subscription for every single event type.

The "types" array in CloudEvents subscription schema can allow a consumer to specify the CloudEvent types eligible to be delivered by the subscription.

This decision also needs to consider the scope issue listed below as (issue4).
**Filters** allow for subscriptions to specify that only a subset of events are to be delivered based on a set of criteria and can be used to request precise types amongst other things. Do we want to include filters for our subscriptions and at some point support advanced filter dialects **(issue 3)**

Scopes (issue 4)

For APIs, the ICM and commonalities WG has clear requirements in terms of defining scopes and managing authorization and authentication to ensure that we have implemented privacy by design thinking.

Currently, we use the same pattern for scopes as defined in API guidelines also for explicit subscriptions. This (might) work if we have a 1-1 relation between subscription and event type.

For e.g. geofencing:subscriptions:write

But if we move to the model where we support multiple event types delivery under a single subscription, it would get tricky. This pattern won't work anymore as the scopes need to reflect subscriptions for every event-type rather than the api-name. What if we have a different scope naming format for explicit subscriptions which would include event-type-short-name instead of api-name? More feedback on this is welcome.

How would opt-out by an end-user at a later point in time (after the subscription was created) impact the subscription lifecycle management? Should we document something in this regard in our guidelines?

Common Configurations for event transmissions (issue 6)

Currently in Camara we use the following config settings:

subscriptionExpireTime
initialEvent (currently requested in an issue)

Some of these configs could be needed by all subprojects and some might only be useful for certain subprojects. Here the cloudevent subscription config field which allows key/value pairs can be useful.

Other points (issue 7 and issue 8)

Do we want to keep the subscription more open already to the possibility of having at some point in time moreprotocol options (as in CloudEvents subscription and possibly make it future proof), or keep it just limited to our current webhook option?
Depending on what we think about some of the above listed questions , we could derive whether we base the subscriptions spec on CloudEvents subscription spec (with our needed changes) or ignore the CloudEvent subscription spec completely.

Possible evolution

If we agree that (any) issue/s listed above is worth discussing, we could split the issues and decide on an aligned a way forward.

Alternative solution

Additional context

id :The unique identifier of the subscription in the scope of the subscription manager.
source: Indicates the source to which the subscription is related.
types: Array of Strings - array of CloudEvents type values
config: A set of key/value pairs that modify the configuration of of the subscription related to the event generation process.
filters: An array of filter expressions that evaluates to true or false.
sink:The address to which events MUST be sent.
sinkCredential: A set of settings carrying credential information that is enabling the entity delivering events to the subscription target to be authorized for delivery at the sink endpoint.
protocol: Identifier of a delivery protocol.
protocolsettings: A set of settings specific to the selected delivery protocol provider.

https://github.com/cloudevents/spec/blob/main/subscriptions/spec.md

Here are some (abstract) examples:

{ 
 "id": "123654", 
 "source": "https://notificationsendserver12.supertelco.com/", 
 "type": "org.camaraproject.device.v0.roaming-status", 
 "specversion": "1.0",
 "status_types": ["ROAMING_ON", "ROAMING_OFF", "COUNTRY_CHANGE"]
 "datacontenttype": "application/json", 
 "data": { 
    "statusType: ROAMING_ON,
    "device": { "phoneNumber": 123456789 },
    "countryCode": 208, 
    "countryName": "FR"
    "subscriptionId": "qs15-h556-rt89-1298"
  },
  "time": "2023-01-17T13:18:23.682Z" 
}

{ 
"id": "83a0d986-0866-4f38-b8c0-fc65bfcda452", 
"source": "https://notificationsendserver12.supertelco.com/", 
"type": "org.camaraproject.device.v0.connectivity-status", 
"specversion": "1.0",
"status_types: ["CONNECTIVITY_DISCONNECTED", "CONNECTIVITY_DATA", "CONNECTIVITY_SMS"],
"datacontenttype": "application/json", 
"data": { 
   "statusType": "CONNECTIVITY_DATA",
   "device": { "phoneNumber": 123456789 },
   "subscriptionId": "qs15-h556-rt89-1298"
 } 
 "time": "2023-01-19T13:18:23.682Z"
}

{
  "id": "56473905-b8c0-fc65bfcda452", 
  "protocol": "HTTP",
  "protocolSettings": {},
  "sink": "https://endpoint.example.com/webhook",
  "sinkCredential": {
    "credentialType": "PLAIN"
  },
  "source": ""https://xyz.abc.com/"",
  "types": [
    "org.camaraproject.device.v0.roaming-status",
    "org.camaraproject.device.v0.connectivity-status"   
  ],
  "filters": [      
     "any": [
    { "exact": { "type": "org.camaraproject.device.v0.roaming-status" , "status_types": ["ROAMING_ON"] } },
    { "exact": { "type": "org.camaraproject.device.v0.connectivity-status", "status_types": ["CONNECTIVITY_DATA"]  } }
  ]
 ], 
"config": { 
    "subscriptionExpireTime": "2023-01-17T13:18:23.682Z" 
} }

MarkCornall commented 4 months ago

Whatever is decided we need to remember that the APIs need to support federation and are interoperable. If a parameter is optional then it needs to be supported by EVERYONE.

bigludo7 commented 4 months ago

Thanks @shilpa-padgaonkar This is an excellent consolidation.

Yes now let's have specific issue to discuss each topic and get feedback from our community.

As said during the meeting, in the issue 6 we can mention https://github.com/camaraproject/Commonalities/issues/140 issue as we have already there some comments.

Thanks !

Ludovic

camaraproject / Commonalities