BottlecapDave / HomeAssistant-OctopusEnergy

Unofficial Home Assistant integration for interacting with Octopus Energy
https://bottlecapdave.github.io/HomeAssistant-OctopusEnergy/
MIT License
535 stars 51 forks source link

3.0.0 & 3.0.1 not gettign data #33

Closed Smiley1244 closed 2 years ago

Smiley1244 commented 2 years ago

Hi love the intergration, I was doing this manually before but so much easier now, thank you.

2.0.0 worked perfectly and was downloading data each day. Since 3.0.0 no data is ever downloaded and sensors stay at 0

Rolling back to 2.0.0 gets data again upgrading to 3.0.1 (or 3.0.0) breaks the intergration

I'm not sure they are relevant but her are the only log entries I can see that are related File "/config/custom_components/octopus_energy/__init__.py", line 89, in async_update_data hass.data[DOMAIN][DATA_RATES] = await client.async_get_rates(tariff_code, period_from, period_to) File "/config/custom_components/octopus_energy/api_client.py", line 107, in async_get_rates return await self.async_get_standard_rates(product_code, tariff_code, period_from, period_to) File "/config/custom_components/octopus_energy/api_client.py", line 49, in async_get_standard_rates data = await response.json(content_type=None) File "/usr/local/lib/python3.9/site-packages/aiohttp/client_reqrep.py", line 1119, in json return loads(stripped.decode(encoding)) File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

May be relevant that i'm an octopus energy UK customer and don't have a varible tarrif.

With 3.0.1 (and 3.0.0) the intergration is getting the following; sensor.octopus_energy_electricity_XXXXXXXXX_latest_consumption = Unavailable sensor.octopus_energy_electricity_XXXXXXXXX_previous_accumulative_consumption = 0 sensor.octopus_energy_electricity_XXXXXXXXX_previous_accumulative_consumption_cost = 0.0 sensor.octopus_energy_electricity_XXXXXXXXX_previous_accumulative_cost = 0 sensor.octopus_energy_electricity_current_rate = 0.187845 (always the same) sensor.octopus_energy_electricity_previous_rate = 0.187845 (always the same) sensor.octopus_energy_gas_XXXXXXXXX_latest_consumption = Unavailable sensor.octopus_energy_gas_XXXXXXXXX_previous_accumulative_consumption = 0 sensor.octopus_energy_gas_XXXXXXXXX_previous_accumulative_consumption_cost = 0.0 sensor.octopus_energy_gas_XXXXXXXXX_previous_accumulative_cost = 0

With 2.0.0 the intergraion get the following: sensor.octopus_energy_electricity_XXXXXXXXX_latest_consumption = 0 sensor.octopus_energy_electricity_XXXXXXXXX_previous_accumulative_consumption = 16.962 sensor.octopus_energy_electricity_XXXXXXXXX_previous_accumulative_consumption_cost = 0.0 sensor.octopus_energy_electricity_XXXXXXXXX_previous_accumulative_cost = Unavailable sensor.octopus_energy_electricity_current_rate = 0.187845 (always the same) sensor.octopus_energy_electricity_previous_rate = 0.187845 (always the same) sensor.octopus_energy_gas_XXXXXXXXX_latest_consumption = 0 sensor.octopus_energy_gas_XXXXXXXXX_previous_accumulative_consumption = 2.781 sensor.octopus_energy_gas_XXXXXXXXX_previous_accumulative_consumption_cost = 0.0 sensor.octopus_energy_gas_XXXXXXXXX_previous_accumulative_cost = Unavailable

Happy to test anything that might help.

BottlecapDave commented 2 years ago

Hello. Sorry to see you're having issues.

The latest_consumption sensors have been removed as part of this release (as indicated in the release notes), as they were always zero due to Octopus Energy never having live data.

I have seen errors crop up when requesting data from Octopus Energy, but this is usually the result either the Octopus Energy API failing or the Home Assistant instance losing internet connection. However, the integration should stabilize itself once a connection has been re-established.

I'll try and do an update which tries to get a better error message.

I also removed some logic around falling back to a variable tariff if an active tariff couldn't be found, however I'd expect a different error message in the logs.

Smiley1244 commented 2 years ago

Thanks, understood about the sensors. I just included everything that might be helpful. Other than reboots for hacs updates I don't think I've had any internet issues. 3.0.1 was running for I think 3/4 days before I rolled back, previous consumption always read 0.

I'll reinstall (Inc any updates) and monitor the logs.

Smiley1244 commented 2 years ago

Hi, I installed 3.0.2 as soon as i saw it but didn't get any results, look this morning and found some log entries. There appears to be 2, one for the usage and 1 for the rates but I have noticed that the rates have registered a change in home assisatant since i updated. do the rates poll more than the usage?

I do notice in both results;

502 Bad Gateway nginx

But I don't have Nginx add-on installed?

502 Bad Gateway nginx

Traceback (most recent call last): File "/config/custom_components/octopus_energy/api_client.py", line 302, in async_read_response return json.loads(text) File "/usr/local/lib/python3.9/json/init.py", line 346, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 187, in _async_refresh self.data = await self._async_update_data() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 147, in _async_update_data return await self.update_method() File "/config/custom_components/octopus_energy/init.py", line 81, in async_update_data tariff_code = await async_get_current_agreement_tariff_code(client, config) File "/config/custom_components/octopus_energy/init.py", line 54, in async_get_current_agreement_tariff_code account_info = await client.async_get_account(config[CONFIG_MAIN_ACCOUNT_ID]) File "/config/custom_components/octopus_energy/api_client.py", line 25, in async_get_account data = await self.async_read_response(response, url) File "/config/custom_components/octopus_energy/api_client.py", line 304, in __async_read_response raise Exception(f'Failed to extract response json: {url}; {text}') Exception: Failed to extract response json: https://api.octopus.energy/v1/accounts/[AccountNumber]; 502 Bad Gateway 502 Bad Gateway nginx

2021-12-29 05:00:39 ERROR (MainThread) [custom_components.octopus_energy.api_client] Failed to extract standard rates: https://api.octopus.energy/v1/products/[TARRIF]/electricity-tariffs/[TARRIF]/standard-unit-rates?period_from=2021-12-29T00:00:00Z&period_to=2021-12-31T00:00:00Z 2021-12-29 05:00:39 ERROR (MainThread) [custom_components.octopus_energy] Unexpected error fetching rates data: Failed to extract response json: https://api.octopus.energy/v1/products/[TARRIF]-21-05-29/electricity-tariffs/[TARRIF]/standard-unit-rates?period_from=2021-12-29T00:00:00Z&period_to=2021-12-31T00:00:00Z;

502 Bad Gateway 502 Bad Gateway</h1></center> nginx Traceback (most recent call last): File "/config/custom_components/octopus_energy/api_client.py", line 302, in __async_read_response return json.loads(text) File "/usr/local/lib/python3.9/json/__init__.py", line 346, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 187, in _async_refresh self.data = await self._async_update_data() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 147, in _async_update_data return await self.update_method() File "/config/custom_components/octopus_energy/__init__.py", line 89, in async_update_data hass.data[DOMAIN][DATA_RATES] = await client.async_get_rates(tariff_code, period_from, period_to) File "/config/custom_components/octopus_energy/api_client.py", line 104, in async_get_rates return await self.async_get_standard_rates(product_code, tariff_code, period_from, period_to) File "/config/custom_components/octopus_energy/api_client.py", line 48, in async_get_standard_rates data = await self.__async_read_response(response, url) File "/config/custom_components/octopus_energy/api_client.py", line 304, in __async_read_response raise Exception(f'Failed to extract response json: {url}; {text}') Exception: Failed to extract response json: https://api.octopus.energy/v1/products/[TARRIF]/electricity-tariffs/[TARRIF]/standard-unit-rates?period_from=2021-12-29T00:00:00Z&period_to=2021-12-31T00:00:00Z; <html> 502 Bad Gateway 502 Bad Gateway nginx Rolling back to 2.0.0 instanly pulls down todays (yesterdays) data. I hope this sheds some light on what is going on and it helps you debug it. Again happy to try anythign or collect data you might need. I don't think there is anything out of the ordinary in my setup and my internet is rock solid. HA is running in a Proxmox VM. Thanks for looking at this though, I do appreciate it. </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/BottlecapDave"><img src="https://avatars.githubusercontent.com/u/14993950?v=4" />BottlecapDave</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>do the rates poll more than the usage?</p> </blockquote> <p>Both poll within HA every minute to make sure they have <em>some</em> data. However at the moment rates do poll more frequently than usage (although there's no real reason why they should as they don't update that frequently). For rates I try and get two days worth of data (today and tomorrow), as this is as far Octopus publishes them for their agile tariffs. </p> <p>Because I can only get historic data for usage, I poll Octopus less frequently (unless the data is out of date).</p> <blockquote> <p>But I don't have Nginx add-on installed?</p> </blockquote> <p>This is the response from Octopus's APIs as they must be using Nginx.</p> <p>I can't see anything in those logs around consumption, so not sure what's causing that to not populate.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Smiley1244"><img src="https://avatars.githubusercontent.com/u/27277726?v=4" />Smiley1244</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>I'll keep 3.0.2 installed for a few days and see if I can find logs for consumption. It's just weird that 2.0.0 always works :)</p> <p>i'll update when I find something, thanks for looking</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Smiley1244"><img src="https://avatars.githubusercontent.com/u/27277726?v=4" />Smiley1244</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>Intrestingly installing 3.0.2 resets the todays data (previously got by 2.0.0) but there are no log events,</p> <p>i'll keep watchign the logs and update if I find anything</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Smiley1244"><img src="https://avatars.githubusercontent.com/u/27277726?v=4" />Smiley1244</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>Same kind of error showed up for consumption, again I've stripped out the HTML around the bad gateway.</p> <p>2021-12-30 03:30:16 ERROR (MainThread) [custom_components.octopus_energy.sensor] Unexpected error fetching rates data: Failed to extract response json: <a href="https://api.octopus.energy/v1/electricity-meter-points/[ID]/meters/[ACCOUNT]/consumption?period_from=2021-12-29T00:00:00Z&period_to=2021-12-30T00:00:00Z">https://api.octopus.energy/v1/electricity-meter-points/[ID]/meters/[ACCOUNT]/consumption?period_from=2021-12-29T00:00:00Z&period_to=2021-12-30T00:00:00Z</a>; 502 Bad Gateway 502 Bad Gateway nginx Traceback (most recent call last): File "/config/custom_components/octopus_energy/api_client.py", line 302, in <strong>async_read_response return json.loads(text) File "/usr/local/lib/python3.9/json/</strong>init<strong>.py", line 346, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 187, in _async_refresh self.data = await self._async_update_data() File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 147, in _async_update_data return await self.update_method() File "/config/custom_components/octopus_energy/sensor.py", line 60, in async_update_data data = await client.async_electricity_consumption(identifier, serial_number, period_from, period_to) File "/config/custom_components/octopus_energy/api_client.py", line 115, in async_electricity_consumption data = await self.</strong>async_read_response(response, url) File "/config/custom_components/octopus_energy/api_client.py", line 304, in __async_read_response raise Exception(f'Failed to extract response json: {url}; {text}') Exception: Failed to extract response json: <a href="https://api.octopus.energy/v1/electricity-meter-points/[ID]/meters/[ACCOUNT]/consumption?period_from=2021-12-29T00:00:00Z&period_to=2021-12-30T00:00:00Z">https://api.octopus.energy/v1/electricity-meter-points/[ID]/meters/[ACCOUNT]/consumption?period_from=2021-12-29T00:00:00Z&period_to=2021-12-30T00:00:00Z</a>; 502 Bad Gateway 502 Bad Gateway nginx</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/BottlecapDave"><img src="https://avatars.githubusercontent.com/u/14993950?v=4" />BottlecapDave</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>I've released a new version <a href="https://github.com/BottlecapDave/HomeAssistant-OctopusEnergy/releases/tag/v3.0.3">v3.0.3</a> which will hopefully supress the 502 bad gateway errors and help us get to the root cause of the issue.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Smiley1244"><img src="https://avatars.githubusercontent.com/u/27277726?v=4" />Smiley1244</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>Thanks, i'll update and let you know what it does.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Smiley1244"><img src="https://avatars.githubusercontent.com/u/27277726?v=4" />Smiley1244</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>First impressions don't look good. It cleared the sensors down on install but left them at zero on reboot. 2.0.0 always gets the last figures on install and reboot. I'll leave running and see if there are any log entries.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/BottlecapDave"><img src="https://avatars.githubusercontent.com/u/14993950?v=4" />BottlecapDave</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>Ok I'll continue to investigate. Out of interest, if you view the details of the accumulative sensor that is not being picked up, I assume it's still associated with the integration (if not, HA will indicate that it has been orphaned and give you the option to remove the entity)</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/BottlecapDave"><img src="https://avatars.githubusercontent.com/u/14993950?v=4" />BottlecapDave</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>The only thing I can see that might be effecting you is if you are missing any 30 minute blocks in your consumtion data. This can be confirmed by going to your energy dashboard on Octopus Energy (<a href="https://octopus.energy/dashboard/new/accounts/A-XXXXXXXX/consumption/home">https://octopus.energy/dashboard/new/accounts/A-XXXXXXXX/consumption/home</a>), switching to day view and seeing if you have any 30 minute periods that are zero?</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Smiley1244"><img src="https://avatars.githubusercontent.com/u/27277726?v=4" />Smiley1244</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>Sensors are not orphaned and show updates to both 'Last Changed' and 'Last Updated' of the upgrade time but value is always 0. I've had no log entries today since installing 3.0.3 but I'll keep monitoring. I had a WiFi sensor drop out of DNS so the log got a bit full today.</p> <p>I don't have any missing blocks for electricity but gas is very patchy, with only 6 blocks over the 24hrs having data but I believe thisto be normal. But again if I install 2.0.0 it downloads the data fine and I get values for electricity and gas. Fresh installs with 2.0.0 always return data, upgrading to anything newer then resets the values to 0. Somethign i haven't done yet but will today is to reset and fresh install 3.0.0+ with no instance of 2.0.0 already running, I'll try that with 3.0.3.</p> <p>If i'm the only person experiencing this I'll continue to run 2.0.0 but I fear I'll miss out on any advancements you make :)</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Smiley1244"><img src="https://avatars.githubusercontent.com/u/27277726?v=4" />Smiley1244</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>and again, thank you for your time spent looking at this.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Smiley1244"><img src="https://avatars.githubusercontent.com/u/27277726?v=4" />Smiley1244</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>So we might have a bit of good news. I uninstalled 3.0.3 completely and rebooted the server. Removed the left over sensors and rebooted again. Installed 3.0.3 and I have data?! :)</p> <p>Only log entries are; <code>/config/custom_components/octopus_energy/sensor.py:65: RuntimeWarning: coroutine 'async_get_active_tariff_code' was never awaited if async_get_active_tariff_code(point["agreements"], client) != None: RuntimeWarning: Enable tracemalloc to get the object allocation traceback /config/custom_components/octopus_energy/sensor.py:73: RuntimeWarning: coroutine 'async_get_active_tariff_code' was never awaited if async_get_active_tariff_code(point["agreements"], client) != None: RuntimeWarning: Enable tracemalloc to get the object allocation traceback</code></p> <p>I continue to monitor and see if it continues to get data over the next few days. It would seem it was the upgrade process from 2.0.0 to 3+ at the root of it?</p> <p>If I can provide anythign that might help diagnose then please let me know.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/BottlecapDave"><img src="https://avatars.githubusercontent.com/u/14993950?v=4" />BottlecapDave</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>So we might have a bit of good news. I uninstalled 3.0.3 completely and rebooted the server. Removed the left over sensors and rebooted again. Installed 3.0.3 and I have data?! :)</p> </blockquote> <p>Not sure why that would have happened, but glad you're getting some data now.</p> <blockquote> <p>Only log entries are;</p> </blockquote> <p>Those logs look like something from the old version of the integration, so are probably safe to ignore.</p> <p>Hopefully we'll have solved this issue.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/BottlecapDave"><img src="https://avatars.githubusercontent.com/u/14993950?v=4" />BottlecapDave</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>I also made an incorrect assumption that we should wait for an entry for each 30 minute slot to be available before making the consumption available, which by the sounds of it will never be true for you with gas. Therefore I'm in the process of removing this assumption.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Smiley1244"><img src="https://avatars.githubusercontent.com/u/27277726?v=4" />Smiley1244</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>(All installs/upgrades were done through HACS) So just for kicks and giggles, i cleanly removed 3.0.3 and did an extra reboot then installed 2.0.0 (all working) again, upgraded to 3.0.3 and . . . no data! Stripped everything off again, extra reboot and installed 3.0.3 and DATA?! </p> <p>So (for me at least) it's somethign about the upgrade process. I'll try an upgrade from a working 3.0.3 with the next release but if that fails to then I'll just cleanly install each time. Hopefully this is somethign unique to my setup and others won't see it BUT if they do then hopefully they will come across this.</p> <p>So again thank you, I'll give it a couple of days and check the logs but if all keeps working I'll close of the issue as user error ;)</p> <p>If you ever need data exports or anythign like that I might be able to help with then just ask.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/Smiley1244"><img src="https://avatars.githubusercontent.com/u/27277726?v=4" />Smiley1244</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>2 days of data for both electricity and gas with 3.0.4.</p> <p>No idea why upgrading from 2.0.0 was breaking it?! Thanks again for looking at the issue.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/BottlecapDave"><img src="https://avatars.githubusercontent.com/u/14993950?v=4" />BottlecapDave</a> commented <strong> 2 years ago</strong> </div> <div class="markdown-body"> <p>Glad that it's now working for you, and hopefully everything will go smoother in the future.</p> </div> </div> <div class="page-bar-simple"> </div> <div class="footer"> <ul class="body"> <li>© <script> document.write(new Date().getFullYear()) </script> Githubissues.</li> <li>Githubissues is a development platform for aggregating issues.</li> </ul> </div> <script src="https://cdn.jsdelivr.net/npm/jquery@3.5.1/dist/jquery.min.js"></script> <script src="/githubissues/assets/js.js"></script> <script src="/githubissues/assets/markdown.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/highlight.min.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/languages/go.min.js"></script> <script> hljs.highlightAll(); </script> </body> </html>