regular dropout pattern rooftop solar QLD

[bdii]

Version of the custom_component

latest version 2022.02.2

Configuration

Raspberry Pi HA OS etc latest versions

Describe the bug

Since the last update I am now receiving rooftop solar data, but there has been what looks like a regular pattern of zero values returned, particularly in the early morning and late afternoon.

It looks like that for a while every third or so api call regularly returns a zero value for rooftop solar, but not for the other attributes. It does not look as though the Nem watch site has the same pattern, ie I have not seen a zero value there when the Opennem in home assistant shows a zero value for rooftop solar.

[bacco007]

This feels like wack-a-mole at the moment!

[bacco007]

I think this is an issue with the OpenNEM API - I'll keep monitoring though

[bacco007]

I've published a beta version that changes the logic around handling the rooftop solar that with any luck should be more reliable.

I've been testing it for a 24 hours, so far so good - I'll test it a bit longer, but if you want to try it out yourself, release is linked below

https://github.com/bacco007/sensor.opennem/releases/tag/2022.02.3b1

[bdii]

I downloaded and installed the beta version this morning, at present it is working fine, but the issue was sporadic, occurring almost all day yesterday for me, but only one aberrant zero value the day before.

At the risk of creating too many more moles to wack:

Since the last update the generation v demand figure is peaking during the day at unrealistic levels. I think this is due to the absence of the solar rooftop production from the demand figure. I think that this corresponds to the demand component not seen by AEMO. See NEM watch figures, there rooftop solar is added to both demand and to production (as unseen by AEMO) components.

For QLD, the following sensors are so far consistently appearing as "unknown" since before the last update,

brown coal - OK , no worries here, as far as I know no brown coal generation in Queensland.

bioenergy biogas - I think that this is from landfill and there is some on the QLD grid.

distillate - some small generators using diesel (and kerosene?) on the QLD grid.

gas steam - not sure whether there are any of these are on the QLD grid not included in the waste gas or biogas categories.

gas recip - a couple of small plants using coal seam methane.

I am not a computer programmer so unfortunately I am unable to help much, if at all, with the programming side, but I very much appreciate that you are working on this integration again and more than happy to beta test.

[bdii]

Having further investigated the Opennem website has not reported gas - steam or bioenergy - biogas data for QLD for at least a year, no gas recip for at least 30 days, and no distillate in the last 7 days. None of these are appearing as attributes of the sensor, so probably there is indeed no data to report recently.

it is possible that the bioenergy - biogas (which I am assuming is landfill gas) is reported as gas - recip as those facilities that i have been able to easily find are recip engines.

However, clearly this is getting down to the minutiae of the details and contributes only a minor amount to the big picture which is black coal and gas on the one hand and solar, wind and hydro on the other for Queensland. In the end the integration can only report what the source(s) provide(s) and it is up to the user to intepret. which given the complexity of the data structures driven the requirements of the different organisations involved (generation v environmental) creates an issue for the relatively naive user like myself.

[bacco007]

None of these are appearing as attributes of the sensor, so probably there is indeed no data to report recently.

I've coded the sensor to only show the generators that operate in a particular market - if those sources start getting reported by the OpenNEM API, they'll automatically drop into the attributes

Since the last update the generation v demand figure is peaking during the day at unrealistic levels. I think this is due to the absence of the solar rooftop production from the demand figure. I think that this corresponds to the demand component not seen by AEMO. See NEM watch figures, there rooftop solar is added to both demand and to production (as unseen by AEMO) components.

I tried to do some reading on this and I have to admit I dont quite understand it - NEM Watch seem to assume that anything generated by rooftop solar is both demand and production, this seems a bit odd to me (on face value).

I could add this functionality if there is broader interest in it.

[bdii]

 The solar data looks good so far.

I understand the logic on the attributes.

I think that the situation with rooftop solar is that it is indeed both generation and demand as it is not measured by the AEMO. My current understanding is as below.

It is not produced by one of the grid generators as it is distributed across the network so is not recorded “officially” as part of the NEM generation. The solar rooftop generation is only “seen” by the grid as a reduced demand over expected demand based on historical trends, and because it is all used locally (partly behind the meter) the same for the demand itself. Basically all the rooftop solar is generated and used without being officially measured by the AEMO. Thus it is added both to the generation and demand figures as generation (and imports) have to match demand (and exports) pretty well.

Hopefully this is clearer.

Sent from my iPad

On 16 Feb 2022, at 17:54, Thomas Baxter @.***> wrote:



None of these are appearing as attributes of the sensor, so probably there is indeed no data to report recently.

I've coded the sensor to only show the generators that operate in a particular market - if those sources start getting reported by the OpenNEM API, they'll automatically drop into the attributes

Since the last update the generation v demand figure is peaking during the day at unrealistic levels. I think this is due to the absence of the solar rooftop production from the demand figure. I think that this corresponds to the demand component not seen by AEMO. See NEM watch figures, there rooftop solar is added to both demand and to production (as unseen by AEMO) components.

I tried to do some reading on this and I have to admit I dont quite understand it - NEM Watch seem to assume that anything generated by rooftop solar is both demand and production, this seems a bit odd to me (on face value).

I could add this functionality if there is broader interest in it.

— Reply to this email directly, view it on GitHubhttps://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fbacco007%2Fsensor.opennem%2Fissues%2F15%23issuecomment-1041210449&data=04%7C01%7C%7C6d039bdbc9754543112e08d9f1218681%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637805948568202279%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=%2Ftap6SatLVyE0GO3OIDR7tljZoDorvO3RgwP3TJaTE0%3D&reserved=0, or unsubscribehttps://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKL6UHNAY6TVH7S4DJJ7ETTU3NJ2LANCNFSM5OGD7QVQ&data=04%7C01%7C%7C6d039bdbc9754543112e08d9f1218681%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637805948568202279%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=ILJj9YO6kRZUIFQCozwciuUkIc6fjjEvBnip2hJ7D1U%3D&reserved=0. Triage notifications on the go with GitHub Mobile for iOShttps://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7C%7C6d039bdbc9754543112e08d9f1218681%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637805948568202279%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Khev62c6NLw0RxhMH03ixHE75NDbOS3Hy0qBSSSOztk%3D&reserved=0 or Androidhttps://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7C%7C6d039bdbc9754543112e08d9f1218681%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637805948568202279%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=u%2BKgwk9vqtaGRqcDMXGc%2FsHFOvZRicFH6ARoGzPKOLw%3D&reserved=0. You are receiving this because you authored the thread.Message ID: @.***>