Decision Proposal 288 - Non-Functional Requirements Revision

[CDR-API-Stream]

This decision proposal contains a proposal for changes to Non-Functional Requirements and the Get Metrics end point based on feedback received through various channels.

The decision proposal is embedded below: Decision Proposal 288 - Non-Functional Requirements Revision.pdf

Consultation on this proposal will close on the 7th April 2023.

Note that the proposed changes have been republished in this comment and this comment

Consultation will be extended to obtain feedback on these updated changes until the 5th May 2023 12th May 2023 19th May 2023.

[ACCC-CDR]

Ecosystem metrics data exposed by participants via the GetMetrics API is published by the ACCC on the CDR Performance Dashboard. Several of the changes proposed here will result in breaking changes for the Performance Dashboard, including breaking continuity of historical data and the ability to compare metrics between versions, which is a mandatory requirement for us to operate and regulate the system. While breaking changes are sometimes necessary to advance the API and enrichen metrics data over time, the averageTps and peakTps overall values need to be maintained in the metrics payload to facilitate this continuity. As the CDR ecosystem grows, our experience shows that it is impractical for all participants to transition between API versions in unison. A carefully constructed transition plan will therefore be necessary to ensure CDR metrics data relating to all participants remains publicly available. Given that we will be responsible for making the necessary changes to the Performance Dashboard and already work closely with participants, we will take the lead on this transition. Retirement dates for v3 should not be set until we have developed a transition plan.

In reference to the JSON schemas for v4 and v5 as posted in the above comment, we provide the following feedback:

• If the plan is to introduce additional granularity into the metrics provided, we need to ensure that the overall values are retained for continuity of historical data.
• As previously stated in our comment above we would like to split the performance metric by performance tier (i.e. Unauthenticated, High Priority, Low Priority, Unattended etc.)
• Providing an aggregated performance metric for all authenticated calls does not provide enough granularity to assess the stated NFR.

[CDR-API-Stream]

Thanks everyone for the final feedback. This thread will now be locked and responses will be posted and a final decision created for submission to the chair.

Note that the final decision on TPS thresholds may take a few more days as the DSB have been offered additional data that may influence the specifics of the thresholds to be set.

[CDR-API-Stream]

Response to feedback on low latency data clusters:

• There is general consensus that this approach is appropriate
• There hasn't been any feedback seeking a change to the current proposal

In response to the question from AEMO: yes, the current proposal would allow for each page of usage history to be called up to 10 times per day. If the threshold needs to be adjusted based on actual experience that can be consulted on in the future. In the interim this should allow for calls being made every couple of minutes to be managed to protect core systems

[CDR-API-Stream]

Response to feedback on changes to the Get Metrics API:

Additional Changes

• As per the request from the ACCC, we will add in fields for the existing aggregated metrics for the purposes of trend continuity. This will be in both v4 and v5 of the Get Metrics API
• As per the request from the ACCC, instead of splitting performance metrics between authenticated and unauthenticated we will these metrics by tier. A metric for the aggregated NFR of 99.5% of all invocations within performance requirements will be retained. This will be included in v5 of the Get Metrics API

Implementation Considerations

• In response to the request for a later implementation date for the Get Metrics changes we will propose retaining the existing date for v4 and pushing back the date for v5 by one milestone. The reason for retaining the v4 date is so that tranche 3 energy retailers going liver at the end of this will be reporting data consistently in the new format from the beginning but also to align to the feedback received that it would be better to begin getting the improved data earlier rather than later
• A deprecation date for v3 of Get Metrics will not be set at this time. The ACCC have indicated they will provide a schedule for their adoption of v4 and v5. Once this is done a deprecation date for v3 will be able to be set.

Responses to other feedback

Some participants suggested other mechanisms for providing data rather than updating the Get Metrics API. We have not modified our proposal in response to this feedback for two reasons:

1. We were specifically requested for improvements to the Get Metrics API by the regulator to assist in the ongoing management of the ecosystem and to reduce the cost of ad hoc reporting. These changes are in line with that request
2. Reception of suggestions related to the voluntary provision of data is influenced by the lack of response to such requests for data by the DSB in the past. It is not clear that a mechanism for voluntary provision of data would be effective

The suggestion that authorisation abandonment metrics should be aligned to software product has not be incorporated into the proposal. Doing so would increase implementation costs but would also be of minimal value as the proposed metrics only come into play once software product process has successfully completed (ie. the customer has already accepted the proposed consent presented by the ADR). The metrics are only representative of the data holder screens which are common across all software products. We may consider this feedback in the future if the concept of data recipient metrics is introduced to CDR.

It was suggested that energy retailers should not be required to provide metrics until the 2 year point of implementation. This is not consistent with the requirements applied to the banking sector in the past but, more importantly, is not a question being considered in this consultation so no action is being taken on this feedback.

[CDR-API-Stream]

The DSB has received data from multiple banks related to the number of active authorisations and TPS levels. Each of the banks that provided detailed data ask that it be kept confidential so it will not be published but it does give a stronger evidence foundation for setting the TPS tiering.

As a result of this data it would appear that our initial proposal (which was based only on number of customers) was far too aggressive and should be altered significantly.

The new proposed tiering for site wide authenticated peak TPS will therefore be:

• For Data Holders with 0 to 10,000 active authorisations, 150 peak TPS total across all consumers
• For Data Holders with 10,001 to 20,000 active authorisations, 200 peak TPS total across all consumers
• For Data Holders with 20,001 to 30,000 active authorisations, 250 peak TPS total across all consumers
• For Data Holders with 30,001 to 40,000 active authorisations, 300 peak TPS total across all consumers
• For Data Holders with 40,001 to 50,000 active authorisations, 350 peak TPS total across all consumers
• For Data Holders with 50,001 to 60,000 active authorisations, 400 peak TPS total across all consumers
• For Data Holders with more than 60,000 active authorisations, 450 peak TPS total across all consumers

Note that the implications of this tiering strategy is as follows:

• No existing data holder will have any immediate obligation increase
• The majority of data holders will get a significant reduction in their site wide obligation (most data holders fit inside the lowest two tiers currently)
• Energy retailers will get a significant reduction in obligation. The tranche 3 retailers going live at the end of the year will probably remain in the bottom tier for the medium term
• For ADRs that are planning to migrate screen scarping customers there should now be enough headroom for interacting with the larger banks as the number of consents being established grows

Responses to specific feedback by participants is as follows:

• There is consensus support for a tiering approach to site wide peak TPS thresholds.
• A number of data holders requested that change be made right now in advance of the proposed workshops later this year. Considering the length of time already invested in consulting on this issue and the existing problems being articulated by the ADR community this is not the preferred approach.
• There was a request to exclude energy retailers from these changes. The NFRs in the standards are deliberately cross-sectoral and the intent is for this to remain the case. The reasons for this exclusion seems to be related to ensuring the obligation on energy retailers is not increased. As the proposed changes results in a significant reduction in obligation for energy retailers (including the tranche 3 retailers) these issues seem to be adequately addressed.
• There was a suggestion to change the NFRs for energy C&I customers specifically. This feedback may be important to address but does not seem aligned to the specific issue being consulted on. This feedback will be noted and raised again during the NFR workshops to be held later this year.

[CDR-API-Stream]

The Data Standards Chair has approved Decision 288 attached below: Decision 288 - Non-Functional Requirements Revision.pdf

It is intended that this decision will be published in the standards in v1.25.0 in the next two weeks.

[nils-work]

Standards version 1.25.0 has now been published, incorporating the changes detailed in the decision above.