Closed CDR-API-Stream closed 2 years ago
The following describes the changes proposed and constraints for this issue.
Please refer to #425 to address multi-sector support for Data Recipients
Data holders utilise CDR Register APIs to obtain information about data recipients in the ecosystem.
With the introduction of the energy sector to the CDR, there is a need to redefine the APIs to simplify alignment to multi-sector support.
Along with these changes, there is an opportunity to address outstanding work to align CDR Register APIs to the conventions defined in the Consumer Data Standards. These changes are to allow for the APIs to be more usable and align their usage across multiple sectors.
Currently, data holders retrieve data recipient information from the following endpoints:
• GetDataRecipientsStatus • GetSoftwareProductsStatus • GetDataRecipients
These APIs have <industry>
within the URI path, inferring this call filters by industry / sector. However, data recipients do not currently align with a sector at the entity level, rather they have sector-specific scopes assigned in the CDR Register and this is reflected in the generated SSA.
Therefore, it is not useful to retrieve data recipient and software product information by industry.
This also is reflected in participant statuses. It is expected that a data holder will be able to observe a data recipient and associated software products move through their various states. However, if a data recipient / software product can jump between sectors, data holders will lose this visibility
A new set of CDR Register APIs will be exposed to allow new builds to align to the new CDR Register standards while keeping the original set maintained until a reasonable deprecation schedule has been decided.
The new set of CDR Register APIs will be exposed targeting the energy sector and allow new builds to align to the new CDR Register standards. The previous set of APIs will be maintained until a reasonable deprecation schedule has been decided.
The new set of APIs will have the following differences from the old:
Remove {industry} from the URI path for the following APIs: • GetDataHolderStatus • GetDataRecipientsStatus • GetSoftwareProductsStatus • GetDataRecipients • GetDataHolderBrands • GetSoftwareStatementAssertion
Structure of the response aligns to data, links, and meta requirements as defined in the CDS. Note that this does NOT include the introduction of pagination
Move x-v header from optional to mandatory
Support x-min-v header as per CDS
Specify If-None-Match and ETag request and response headers in the public APIs
Add energy value to industry enumeration
Where sector is returned in the response field will move to an array to reflect multiple sectors
Align response error codes to the CDS • 400 for missing required header / Invalid Version • 406 for unsupported version • 304: Not modified to align to the ETag pattern • 404: Add support as per CDS to handle missing resources
To reiterate, these changes would be implemented against the new set of APIs, with the previous set retained so current implementations remain unaffected. However, an expectation would be set that data holders would eventually migrate to these APIs and a reasonable deprecation schedule would be decided.
Options:
https:// <register-base-path>/ cdr-register / v2 / data-holders / brands
This approach is deemed inappropriate as this value represents the version of the standards, not individual or sets of endpoints.all
and increment the endpoint version number: https:// <register-base-path>/ cdr-register / v1 / all / data-holders / brands
This couples the endpoints together and does not allow for both sets of endpoints to be separately maintainedhttps:// <register-base-path>/ cdr-register / v1 / data-holders / brands
This has the result of the new APIs having unique paths and not clashing with the old. Documentation will be updated to reflect that the new APIs should be targeted for new builds and the old APIs will be maintained until deprecated.Our recommendation is option 3.
The intent of these changes is to make them available for energy builds so no changes are imposed on the banking sector. However, consideration would need to be made to determine when the old APIs are to be deprecated. Setting expectations early on what a reasonable timeframe looks like would aid participant development roadmap planning.
The GetDataHolderBrands API definition evolved to meet the need to store additional metadata for the discovery and authentication of Data Holder Brands.
The GetDataHolderBrands API allows for the following:
This endpoint is only available to authenticated data recipients. This design approach has resulted in the following:
Allowing access to these CDR Register endpoints to authenticated users only adds a layer of obfuscation. Other security mechanisms are present to protect these data holder endpoints, therefore the benefit of securing this information behind an authentication mechanism is being re-examined.
Allowing unauthenticated access at an equivalent endpoint simplifies the usage of this endpoint for change detection. There are additional benefits to exposing this endpoint as a public API and using the same pattern as GetDataRecipients. This simplifies usage and allows clients to leverage ETag and CDN functionality.
The following changes are proposed:
Create an unauthenticated Data Holder Brands API returning the same content as the current authenticated API. The authenticated version of the API will be deprecated once a schedule has been determined
Create an unauthenticated Data Holder Brands API returning a subset of the content currently returned in the authenticated API. Security endpoint information would NOT be included in this API. Both the authenticated and unauthenticated APIs would be maintained.
What are the consequences of making this information available publicly?
With the introduction of the energy sector to the Consumer Data Right, the relationship between data holder brands and sectors needs to be defined.
With the initial roll-out of energy, the current model will NOT change and data holders which span multiple industries will be required to create one brand per industry on the CDR Register.
Further design work is scheduled to examine how multiple industries may be facilitated by one brand on the CDR Register. This work will happen in parallel but is not guaranteed to be complete and available in the initial energy rollout timeframe.
Add documentation describing the constraint of one sector per brand when configuring the Data Holder Brand in the RAAP
The current assumption is the initial data holders in the energy sector will be unaffected by this constraint as their initial rollout will not span multiple sectors.
Add an unauthenticated GetDataHolderBrands endpoint exposed as a public API Further work is required to consider these changes. Issue #444 has been raised to continue this work post maintenance iteration 9.
The rest of the change request will be addressed through maintenance iteration 9.
Upon review, it has been determined that the benefits for option 3, allowing two sets of APIs to be maintained in parallel until one is deprecated, can be equivalently achieved using option 2, incrementing the endpoint version numbers, and allowing multiple endpoint versions to be made available in parallel.
Option 3's original requirement to maintain two separate sets of endpoints independently adds complexity to the versioning model with limited benefit. The constraint with adopting option 2 is, if changes are anticipated in the future to one of the endpoints, participants will be required to migrate to the latest version of that endpoint. This is a minimal constraint as participants will eventually migrate to the latter versions of the endpoints anyway.
Therefore, option 2 will be adopted and the the Endpoint Version Schedule details the different versions of the endpoints which will be made available:
This approach has the advantage of maintaining consistency with the current endpoint versioning mechanism described in the Consumer Data Standards and allows participants to plan endpoint migration accordingly.
Previous endpoint deprecation and retirement dates will be explored in a future maintenance iteration. Issue #452 has been raised to track this work.
This change was incorporated into release v1.15.0. Refer to Decision 212 for further details.
Description
Data holders will need to decide how to structure their brands in the CDR Register to cater for more than one sector. The introduction of energy raises the question on how data holder brands should support different sectors and how the dynamic client registrations generated should map to different sectors
Area Affected
Register APIs will need to be structured to expose single or sector agnostic discovery and status
Dynamic client registration specification will need to define how the registration process will support single and multi-sector scenarios
Change Proposed
Provide requirements to data holders on how a data holder brand will cater to both single and multi-sector scenarios