Closed CDR-API-Stream closed 3 years ago
SelenaLiuEA
commented
3 years ago Good afternoon All,
EnergyAustralia’s initial feedback is below:
Cheers Selena
CDR-API-Stream
commented
3 years ago In response to @SelenaLiuEA:
- For the field meterSerial, please note that multiple meterSerial numbers may exist for a NMI.
We will take this into consideration
- For unitOfMeasure, GWh is not a valid response. We would either support Wh and kwh (and technically Mwh) or to standardise on kwH.
The GWh inclusion was tentative. If there is no usefulness we will take out the unitOfMeasure field and assume kWh
- Other meter data may be useful in potential use cases, such as KVArh which may relate to some demand tariffs.
It isn't clear that this is data held by AEMO (the designated data holder for this data set). Is this data that AEMO would be holding?
johnAEMO
commented
3 years ago Thank you for the opportunity of responding to the above Decision Proposal.
As you are aware AEMO has already assisted in the preparation of this proposal and our feedback is mostly in terms of clarification.
Under Current Recommendation, General Notes: • Readings are sorted in descending date order as stated but also by NMI, then date, then meter and then suffix • Service point is a term already used in the energy industry to genericise numbering between electricity and gas fuels – and translates directly to NMI and MIRN respectively • By “intermingled” we assume that refers to data from different meter types (interval or basic) intermingled in the data feed
Common Object Types: • registerSuffix – in historical data, where this is missing, AEMO will provide the data Stream suffix – therefore this field will be Mandatory • meterSerial – in historical data this may not be present so this field is Optional (in the long term this will be Mandatory) • unitOfMeasure – will never be GWh and always (for the data to be provided) be kWh • aggregateValue – in the description, “If positive then it means net consumption, if negative it means net export from the premise”
Bulk Usage Data: • We assume that “all service points associated with a consumer” are determined at time of authentication, returned to the gateway and used as the basis for data retrieval within the one ‘consumer session’. Either way we would welcome further discussion to confirm or modify this assumption.
Thanks again John A
PratibhaOrigin
commented
3 years ago Payload structure
For ease of data review by customers or recipients , perhaps it is easier to stream/compile each service point from start to finish by date service point by service point then intermingle service points into a date block. Especially if the service points are of different lengths in time – like if someone has only recently added a solar register to their respective existing light and power.
Common Object Types
If the tariff for that service point would also be added to allow the differentiator when the meter serial number is the same but has a different NMI suffix. Like meter 12345 and NMI suffix E1 on Reg 1 and E2 on Reg 2. How is a customer to know the difference between what is measured on E1 or E2. Perhaps should be meter 12345 and NMI suffix E1 on Reg 1 with Tariff L/P and E2 on Reg 2 with Tariff C/L for example.
Read Quality - Why is it optional ? Considering the explanation - when its blank, its supposed to be actual, assume its should be 'conditional' instead of 'optional'.
ReadEnddate - Suggest to make readenddate a mandatory field similar to readstartdate. Instead of assuming the data available for 24 months and using that logic to deduce start and end dates, it will be better to make them both ‘mandatory’ fields.
registerSuffix - Suggest to make registersuffix ‘mandatory’ field instead of ‘optional' so that its clear to use the the usage data in scenarios where there are multiple suffixes for a particular serial number.
ServicePointID -
Scope-
General -
Cheers! Prats
CDR-API-Stream
commented
3 years ago Thanks for all of the feedback. This thread is now closed for further feedback.
We are processing the responses and will provide comment below. Further feedback on this payload (and the rest of the standard) will occur once consultation on the remaining data clusters is complete and we are able to publish a full draft.
CDR-API-Stream
commented
3 years ago In response to John from AEMO:
As you are aware AEMO has already assisted in the preparation of this proposal and our feedback is mostly in terms of clarification.
AEMO's support during this process has been appreciated :)
General Notes: • Readings are sorted in descending date order as stated but also by NMI, then date, then meter and then suffix
Currently only the date order is specified. Additional sorting is left to the holder.
• Service point is a term already used in the energy industry to genericise numbering between electricity and gas fuels – and translates directly to NMI and MIRN respectively
This is one of the reasons for using Service Point as it will allow extension to gas in the future.
• By “intermingled” we assume that refers to data from different meter types (interval or basic) intermingled in the data feed
That is corrcet
Common Object Types: • registerSuffix – in historical data, where this is missing, AEMO will provide the data Stream suffix – therefore this field will be Mandatory • meterSerial – in historical data this may not be present so this field is Optional (in the long term this will be Mandatory) • unitOfMeasure – will never be GWh and always (for the data to be provided) be kWh • aggregateValue – in the description, “If positive then it means net consumption, if negative it means net export from the premise”
Thanks, this feedback will be incorporated.
Bulk Usage Data: • We assume that “all service points associated with a consumer” are determined at time of authentication, returned to the gateway and used as the basis for data retrieval within the one ‘consumer session’. Either way we would welcome further discussion to confirm or modify this assumption.
That is the current working assumption.
CDR-API-Stream
commented
3 years ago In response to feedback from Origin Energy:
Payload structure For ease of data review by customers or recipients , perhaps it is easier to stream/compile each service point from start to finish by date service point by service point then intermingle service points into a date block. Especially if the service points are of different lengths in time – like if someone has only recently added a solar register to their respective existing light and power.
This was the intent of the service point specific end point. If a client wanted to stream data according to service point then they can call the service point specific API
Common Object Types If the tariff for that service point would also be added to allow the differentiator when the meter serial number is the same but has a different NMI suffix. Like meter 12345 and NMI suffix E1 on Reg 1 and E2 on Reg 2. How is a customer to know the difference between what is measured on E1 or E2. Perhaps should be meter 12345 and NMI suffix E1 on Reg 1 with Tariff L/P and E2 on Reg 2 with Tariff C/L for example.
This feedback isn't entirely clear. Currently the payload covers Service Point (ie. NMI), meter ID, meter suffix and meter register. It is understood that this is sufficient.
Read Quality - Why is it optional ? Considering the explanation - when its blank, its supposed to be actual, assume its should be 'conditional' instead of 'optional'.
No. Conditional indicates that the field is only sometimes mandatory from a schema perspective, usually due to the value of a different field. An optional field is one they may or may not be present depending on whether the Data Holder has valid data.
Another pattern that we commonly use in the standards is where a mandatory field is a single value in most cases we will make the field optional but specify a default interpretation if the field is absent. This saves space in the payload but has the same semantic outcome for the client.
ReadEnddate - Suggest to make readenddate a mandatory field similar to readstartdate. Instead of assuming the data available for 24 months and using that logic to deduce start and end dates, it will be better to make them both ‘mandatory’ fields.
Why would this be an improvement? The feedback is welcome but the reasoning is non-obvious.
registerSuffix - Suggest to make registersuffix ‘mandatory’ field instead of ‘optional' so that its clear to use the the usage data in scenarios where there are multiple suffixes for a particular serial number.
This field is optional because AEMO has indicated that it is not always held. In this scenario making it mandatory would mean the data could not be presented in a schema compliant manner in all cases.
ServicePointID -
- More clarity on the definition of ServicepointID - Is it a random key as per ID permanence representing - the combination of ‘ADR - Customer - Data holder’ . Is the Service point ID specific to a NMI ?
Please refer to the discussion for the NMI standing data. The Service Point ID is a one to one map to a NMI and is consistent within the context of customer, ADR and Retailer as per the ID permanence rules currently in the CDR standards.
- Who generates the ServicePointID - Is it AEMO or the data holders(retailer) ? Is service point ID known to data holders and are they supposed to store it similar to AccountID ?
This is an excellent question that we do not yet have an answer for. We will need to get to that level of detail once the standards are completed in draft and we started thinking about implementation considerations.
Scope
- Suggest that for non contestable, unmetered sites – known as NCONUML – whilst some may be SME sites, given that these are calculated loads, not real consumption reads, that they would not really be reaching out to use CDR. Perhaps deem them to be a form of Type 7, who are deemed to be out of scope (pg 2). Please correct if assumption is incorrect.
- Embedded networks and unmetered sites may have specific usage data structure requirements .Example - Off market sites don't have market NMIs. These need to discussed with AEMO and confirm scope for the usage payload accordingly.
Further consultation on this feedback would be very welcome. As it pertains to the designation and eligibility it would be worthwhile discussing this feedback with the ACCC also.
General -
- Different data holders query in the requested period - If the ADR requested data for last 2 years and is with current retailer for only last 3 months, how does AEMO know which are the other retailers/data holders to be contacted for data as well as authorisation purposes before sharing usage data.
At the moment there is no technical solution for obtaining usage beyond the existing retailer/customer relationship as AEMO cannot reliably determine the ownership of a NMI over time using the data it receives. This is a topic being discussed between the ACCC, DSB and AEMO.
- Also worth considering is that we will continue to see basic mater data, 30m data and 15m data for many years to come. Not all meters will be converted to 5m in a big bang. Where a consumer request data for multiple years, it is highly likely that there will be a mix of various granularities of the reads. How does the structure supports the mix of granularities for reads in a period ?
Agreed. The structure of these payloads have been deliberately constructed to allow for this variations. If the structure does not appear to service this scenario then that feedback would be very welcome.
The decision proposal for Electricity Usage data payloads is attached below: Decision Proposal 112 - Usage Data Payloads.pdf
Feedback is welcome on this proposal. This thread will remain open for feedback until the 9th of October.
The DSB would like to thank the AEMO team for their support in developing this proposal. As AEMO is the single data holder for these payloads they have been modelled on data that is actually held by AEMO or can be easily derived.
As previously discussed in various forums, the practice of the DSB is to respond to feedback incrementally before the consultation is complete to promote an interactive style of consultation. Participants are therefore encouraged to provide feedback earlier in the consultation process so that the community can work together to a consensus outcome.