NFR: Policing of Data Recipient Low Velocity Data Sets Thresholds

[perlboy]

Description

At the MI call today I requested clarification and then raised operational issues with respect to the enforcement of the recently introduced Data Recipient data set thresholds for Low Velocity Data Sets. The issue at hand is that currently the Standards state:

Identified low velocity data sets are to be handled according to the following table noting that:

• the Velocity Time Period is a continuous period of time in which calls beyond a specific threshold MAY be rejected by the Data Holder
• the Allowable Call Volume is the threshold number of calls to the same resource for the same arrangement above which calls MAY be rejected by the Data Holder

The issue here is that the MAY component is not a requirement on primary Data Holders and in fact if it was implemented would effectively result in Data Holders being required to police data recipients on their behaviour with respect to calls to the Secondary Data Holder.

Operationally this is a hot topic because the SDH has had numerous availability and data quality issues with Primary Data Holders (or their technology vendors) "stuck" in-between and subjected to accusations when enacting Exemptions to Protect Service to protect infrastructure from abusive recipients. This is exacerbated by very high failure behaviours caused by downstream outages for which the Holders are not in control. It is for this reason the proposal is to convert this to a MUST but move the responsibility to the Secondary Data Holder to appropriately enforce.

During the MI one comment was "how is this different to existing NFRs", some of the reasons are:

• Many (most?) Data Holders do not have a hard enforcement of the NFR, instead relying on soft control by exception under the Exemptions to Protect Service. This works fine for situations where the Data Holder has sufficiently capacity planned for their data source resulting in much higher thresholds than those observed for SDH endpoints.
• The Primary Data Holder NFRs are structured as per/second throughputs not sliding scale measurements which fundamentally changes how measurement and enforcement can be implemented
• For the most part, Primary Data Holders do not have ongoing performance issues nor experience significant outliers (up to 10 seconds for responses) like those experienced when interacting with the SDH
• Data Recipient behaviours on SDH APIs appear to be much more abusive than those observed on other APIs. Certain data recipients are for instance, making thousands of calls in a day to the same NMI requesting the same Service Point data and requesting usage data on an hourly basis
• The technology pattern for Holder to Secondary Holder is unique as it is effectively a proxy/middleware deployment which by definition has with it unique challenges such as north-south socket contention on front channel services and hair-pinning between, at a minimum Holder and SDH but often via middleware (i.e. Holder CDR -> Holder SDH Breakout -> SDH).

Note: I've created this pretty quickly for consideration in this MI and therefore have not had a chance to pass this through Biza.io's Data Standards Committee.

Area Affected

• Non-functional Requirements > Data Recipient Requirements > Low Velocity Data Sets
• Shared Responsibility > Energy > Endpoint Variations

Change Proposed

• Add a statement to Low Velocity Data Sets to the effect of "The Secondary Data Holder MUST utilise the supplied x-cds-arrangement header to enforce the Non-Functional Requirements

• Introduce a new enhanced error behaviour. This is different to 429 as a 429 does not purvey if the constraint is because of the Primary or Secondary Holder

• Alter the Low Velocity Data Sets section to:

• the Velocity Time Period is a continuous period of time in which calls beyond a specific threshold MUST be rejected by the Secondary Data Holder
• the Allowable Call Volume is the threshold number of calls to the same resource for the same arrangement above which calls MUST be rejected by the Secondary Data Holder

• Alter the statement "This field MUST be populated but AEMO MUST NOT seek to validate the consent associated with the arrangement" to "This field MUST be populated but AEMO MUST NOT seek to validate the consent associated with the arrangement other than for non-functional requirement enforcement"

[johnAEMO]

AEMO has reviewed this change and concur that protection of our service is more appropriately undertaken by AEMO.

• We have analysed the impact of throttling based on arrangement and found that throttling of 10%-80% of call volumes would have been made in July (80% of high volumes pre mid July - equating to 5300 calls per day, and 10% of volumes post mid July when volumes decreased significantly - equating to throttling of 170 calls per day) - based on throttling by arrangement for more than 10 (successful) calls to the same API on the same day
• Initial estimates of effort are that such throttling may not be a significant change
• We recognise the concerns that secondary data holder rejections may require a different error code. AEMO has no fixed position on this but consultation would need to be finalised before any throttling changes are made (and if the code for existing secondary data holder throttling would change also) We would recommend that the Low Velocity Data Sets section read: -the Velocity Time Period is a continuous period of time in which calls beyond a specific threshold MAY be rejected by the Secondary Data Holder
• the Allowable Call Volume is the threshold number of calls to the same resource for the same arrangement above which calls MAY be rejected by the Secondary Data Holder