home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
72.63k stars 30.4k forks source link

OPower integration for ConEdison always shows $0 cost from "Opower cned elec 0xxxxxxxxx cost" (consumption works fine) #108551

Open ncd7 opened 9 months ago

ncd7 commented 9 months ago

The problem

I've linked my ConEdison account successfully and I'm seeing the statistics flow through. (I've had the linkage for a few weeks now). The electricity consumpion statistic works fine but the cost one is always 0.0 This looks like a bug to me. Thank you.

What version of Home Assistant Core has the issue?

core-2024.1.4

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

OPower

Link to integration documentation on our website

https://www.home-assistant.io/integrations/opower

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 9 months ago

Hey there @tronikos, mind taking a look at this issue as it has been labeled with an integration (opower) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `opower` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign opower` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)


opower documentation opower source (message by IssueLinks)

ncd7 commented 9 months ago

I see there is a request for logs. I've just turned on "Enable Debug Logging" for the OPower integration. I assume it might take sometime for relevant logs to be generated but pls let me know in the meantime if this is a known issue. I searched on Github but couldn't find anything.

tronikos commented 9 months ago

This is a known issue for many utilities. Many utilities provide cost data only when requesting monthly data and not when requesting hourly or daily data. Need to update the documentation.

ncd7 commented 9 months ago

Ah damn, I see. Yes updating the documentation would be super useful as I checked on there first. Do you think any workarounds are possible? I guess I could look at recent statements and set the fixed price in the energy dashboard?

tronikos commented 9 months ago

No idea if a static price would work. If it does let us know to mention it in the documentation.

kan84 commented 7 months ago

@ncd7 were you able to figure out a workaround? Static price option in energy dashboard for opower seems to be disabled.

ncd7 commented 7 months ago

@kan84 -- unfortunately not really :(

holocronology commented 5 months ago

I'm actually not getting any cost data regardless of day, week, month, etc... on ConEd with the "cost" entity for elec OR gas.

holocronology commented 5 months ago

The option to either set a static price or to use a static price where data from the entity doesn't exist would be even better. In that case it could display as "estimated."

holocronology commented 4 months ago

Still an issue.

tronikos commented 2 months ago

Is this still an issue? The latest release always fetches the last month bill so if your utility provides monthly cost data you might be able to see them now. You will have to switch to the yearly view in the energy dashboard to see monthly costs.

holocronology commented 2 months ago

This is still an issue for me, yes. I have no cost data. Using ConEd in NY.

tronikos commented 2 months ago

Do you have cost data on your utility website? At what granularity? Can you post a screenshot of the graph you see there? In Home Assistant make sure you use the statistics and not the sensors. Can you post a screenshot of the energy dashboard configuration page and the energy dashboard at the yearly view?

holocronology commented 2 months ago

conEdison appears to provide monthly data on the web view: image image

However, I know more granular data (day) is available by download.

There is a Third-Party option to share your data, but obviously OPower is not on this list: image

Here is a configuration page: image

Dashboard: image

I do wish that this was still an option, as I could always put in a price for estimating, but obviously it should be able to pull this data. image

Let me know if I can provide any additional data.

tronikos commented 2 months ago

You have correctly configured the energy dashboard with the statistics. I wanted to confirm you didn't select the sensors. In the energy dashboard of Home Assistant if you change it to the year view and check costs for different years, do you see cost? You should be seeing costs up to 3 years ago and starting again from last month. Can you add this card and post a screenshot:

chart_type: bar
period: month
type: statistics-graph
entities:
  - opower:XXX_elec_YYY_energy_cost
stat_types:
  - change
days_to_show: 10000

Replace XXX and YYY with your values.

holocronology commented 2 months ago

Change to year view: I see no cost data, only usage.

Added card as requested, changing XXX to "cned" and YYYY to "the number from the cost entity (aka account number). I then rebooted the system and let it sit for a little while. No data shows.

image

tronikos commented 2 months ago

Can you run:

pip install opower
python -m opower --aggregate_type day
python -m opower --aggregate_type bill

Do you get cost data when you request bill vs day?

holocronology commented 2 months ago

Can you run:

pip install opower
python -m opower --aggregate_type day
python -m opower --aggregate_type bill

Do you get cost data when you request bill vs day?

Can you let me know where to run that? I can get to terminal but it doesn't work.

tronikos commented 2 months ago

On your laptop or desktop. You need to first install python.

holocronology commented 2 months ago

This is what I am seeing. However the "2fa" secret is the same as the one I used in HA and it is working.

`

PS C:\Windows\system32> python -m opower --aggregate_type day Utility, one of ['aepohio', 'aeptexas', 'appalachianpower', 'atlanticcityelectric', 'bge', 'coautilities', 'comed', 'coned', 'delmarva', 'duquesnelight', 'enmax', 'evergy', 'indianamichiganpower', 'kentuckypower', 'oru', 'peco', 'pepco', 'pge', 'portlandgeneral', 'pse', 'psoklahoma', 'scl', 'smud', 'swepco']: coned Username: Password: 2FA secret: Traceback (most recent call last): File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\opower\opower.py", line 199, in async_login self.access_token = await self.utility.async_login( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\opower\utilities\coned.py", line 70, in async_login async with session.post( File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\aiohttp\client.py", line 1197, in aenter self._resp = await self._coro ^^^^^^^^^^^^^^^^ File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\aiohttp\client.py", line 696, in _request resp.raise_for_status() File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\aiohttp\client_reqrep.py", line 1070, in raise_for_status raise ClientResponseError( aiohttp.client_exceptions.ClientResponseError: 500, message='Internal Server Error', url=URL('https://www.coned.com/sitecore/api/ssc/ConEdWeb-Foundation-Login-Areas-LoginAPI/User/0/Login')

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\opower__main__.py", line 214, in asyncio.run(_main()) File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\asyncio\runners.py", line 194, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\asyncio\runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\asyncio\base_events.py", line 687, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\opower__main__.py", line 93, in _main await opower.async_login() File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\opower\opower.py", line 207, in async_login raise CannotConnect(err) opower.exceptions.CannotConnect: 500, message='Internal Server Error', url=URL('https://www.coned.com/sitecore/api/ssc/ConEdWeb-Foundation-Login-Areas-LoginAPI/User/0/Login')

PS C:\Windows\system32> python -m opower --aggregate_type bill Utility, one of ['aepohio', 'aeptexas', 'appalachianpower', 'atlanticcityelectric', 'bge', 'coautilities', 'comed', 'coned', 'delmarva', 'duquesnelight', 'enmax', 'evergy', 'indianamichiganpower', 'kentuckypower', 'oru', 'peco', 'pepco', 'pge', 'portlandgeneral', 'pse', 'psoklahoma', 'scl', 'smud', 'swepco']: coned Username: Password: 2FA secret: Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\opower__main__.py", line 214, in asyncio.run(_main()) File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\asyncio\runners.py", line 194, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\asyncio\runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\asyncio\base_events.py", line 687, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\opower__main__.py", line 93, in _main await opower.async_login() File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\opower\opower.py", line 199, in async_login self.access_token = await self.utility.async_login( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\admin\AppData\Local\Programs\Python\Python312\Lib\site-packages\opower\utilities\coned.py", line 111, in async_login raise InvalidAuth( opower.exceptions.InvalidAuth: 2FA code was invalid. Is the secret wrong?

`

raidenii commented 2 months ago

Do you get cost data when you request bill vs day?

I tested that but cost column is always 0:

(opower) me@wsl:~$ python -m opower --aggregate_type day
Utility, one of ['coned', 'psoklahoma', 'scl', 'evergy', 'pse', 'bge', 'duquesnelight', 'portlandgeneral', 'appalachianpower', 'oru', 'aeptexas', 'delmarva', 'atlanticcityelectric', 'pge', 'indianamichiganpower', 'aepohio', 'smud', 'peco', 'swepco', 'comed', 'coautilities', 'pepco', 'kentuckypower', 'enmax']: coned
Username: 
Password:
2FA secret: 

Current bill forecast: Forecast(account=Account(customer=Customer(uuid='11111111-2222-3333-4444-555555555555'), uuid='11111111-2222-3333-4444-555555555555', utility_account_id='1234567', id='1234567', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=None), start_date=datetime.date(2024, 6, 27), end_date=datetime.date(2024, 7, 26), current_date=datetime.date(2024, 7, 23), unit_of_measure=<UnitOfMeasure.KWH: 'KWH'>, usage_to_date=730.0, cost_to_date=0.0, forecasted_usage=896.0, forecasted_cost=0.0, typical_usage=0.0, typical_cost=0.0)

Current bill forecast: Forecast(account=Account(customer=Customer(uuid='11111111-2222-3333-4444-555555555555'), uuid='11111111-2222-3333-4444-555555555555', utility_account_id='7654321', id='7654321', meter_type=<MeterType.GAS: 'GAS'>, read_resolution=None), start_date=datetime.date(2024, 6, 27), end_date=datetime.date(2024, 7, 26), current_date=datetime.date(2024, 7, 23), unit_of_measure=<UnitOfMeasure.THERM: 'THERM'>, usage_to_date=6.0, cost_to_date=0.0, forecasted_usage=7.0, forecasted_cost=0.0, typical_usage=0.0, typical_cost=0.0)

Getting historical data: account= Account(customer=Customer(uuid='11111111-2222-3333-4444-555555555555'), uuid='11111111-2222-3333-4444-555555555555', utility_account_id='123456787654321', id='11111111-2222-3333-4444-555555555555', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=<ReadResolution.QUARTER_HOUR: 'QUARTER_HOUR'>) aggregate_type= day start_date= 2024-07-16 20:15:06.716828 end_date= 2024-07-23 20:15:06.716920
start_time      end_time        consumption     provided_cost   start_minus_prev_end    end_minus_prev_end
2024-07-16 00:00:00-04:00       2024-07-17 00:00:00-04:00       51.261  0       None    None
2024-07-17 00:00:00-04:00       2024-07-18 00:00:00-04:00       51.912  0       0:00:00 1 day, 0:00:00
2024-07-18 00:00:00-04:00       2024-07-19 00:00:00-04:00       41.772  0       0:00:00 1 day, 0:00:00
2024-07-19 00:00:00-04:00       2024-07-20 00:00:00-04:00       21.309  0       0:00:00 1 day, 0:00:00
2024-07-20 00:00:00-04:00       2024-07-21 00:00:00-04:00       20.979  0       0:00:00 1 day, 0:00:00
2024-07-21 00:00:00-04:00       2024-07-22 00:00:00-04:00       23.517  0       0:00:00 1 day, 0:00:00
2024-07-22 00:00:00-04:00       2024-07-23 00:00:00-04:00       19.539  0       0:00:00 1 day, 0:00:00
2024-07-23 00:00:00-04:00       2024-07-24 00:00:00-04:00       3.999   0       0:00:00 1 day, 0:00:00

Getting historical data: account= Account(customer=Customer(uuid='11111111-2222-3333-4444-555555555555'), uuid='11111111-2222-3333-4444-555555555555', utility_account_id='123456787654321', id='11111111-2222-3333-4444-555555555555', meter_type=<MeterType.GAS: 'GAS'>, read_resolution=<ReadResolution.HOUR: 'HOUR'>) aggregate_type= day start_date= 2024-07-16 20:15:06.716828 end_date= 2024-07-23 20:15:06.716920
start_time      end_time        consumption     provided_cost   start_minus_prev_end    end_minus_prev_end
2024-07-16 00:00:00-04:00       2024-07-17 00:00:00-04:00       0.06    0       None    None
2024-07-17 00:00:00-04:00       2024-07-18 00:00:00-04:00       0.24    0       0:00:00 1 day, 0:00:00
2024-07-18 00:00:00-04:00       2024-07-19 00:00:00-04:00       0.44    0       0:00:00 1 day, 0:00:00
2024-07-19 00:00:00-04:00       2024-07-20 00:00:00-04:00       0.22    0       0:00:00 1 day, 0:00:00
2024-07-20 00:00:00-04:00       2024-07-21 00:00:00-04:00       0.32    0       0:00:00 1 day, 0:00:00
2024-07-21 00:00:00-04:00       2024-07-22 00:00:00-04:00       0.28    0       0:00:00 1 day, 0:00:00
2024-07-22 00:00:00-04:00       2024-07-23 00:00:00-04:00       0.18    0       0:00:00 1 day, 0:00:00
(opower) me@wsl:~$ python -m opower --aggregate_type bill
Utility, one of ['coned', 'psoklahoma', 'scl', 'evergy', 'pse', 'bge', 'duquesnelight', 'portlandgeneral', 'appalachianpower', 'oru', 'aeptexas', 'delmarva', 'atlanticcityelectric', 'pge', 'indianamichiganpower', 'aepohio', 'smud', 'peco', 'swepco', 'comed', 'coautilities', 'pepco', 'kentuckypower', 'enmax']: coned
Username: 
Password:
2FA secret: 

Current bill forecast: Forecast(account=Account(customer=Customer(uuid='11111111-2222-3333-4444-555555555555'), uuid='11111111-2222-3333-4444-555555555555', utility_account_id='1234567', id='1234567', meter_type=<MeterType.GAS: 'GAS'>, read_resolution=None), start_date=datetime.date(2024, 6, 27), end_date=datetime.date(2024, 7, 26), current_date=datetime.date(2024, 7, 23), unit_of_measure=<UnitOfMeasure.THERM: 'THERM'>, usage_to_date=6.0, cost_to_date=0.0, forecasted_usage=7.0, forecasted_cost=0.0, typical_usage=0.0, typical_cost=0.0)

Current bill forecast: Forecast(account=Account(customer=Customer(uuid='11111111-2222-3333-4444-555555555555'), uuid='11111111-2222-3333-4444-555555555555', utility_account_id='7654321', id='7654321', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=None), start_date=datetime.date(2024, 6, 27), end_date=datetime.date(2024, 7, 26), current_date=datetime.date(2024, 7, 23), unit_of_measure=<UnitOfMeasure.KWH: 'KWH'>, usage_to_date=730.0, cost_to_date=0.0, forecasted_usage=896.0, forecasted_cost=0.0, typical_usage=0.0, typical_cost=0.0)

Getting historical data: account= Account(customer=Customer(uuid='11111111-2222-3333-4444-555555555555'), uuid='11111111-2222-3333-4444-555555555555', utility_account_id='123456787654321', id='11111111-2222-3333-4444-555555555555', meter_type=<MeterType.ELEC: 'ELEC'>, read_resolution=<ReadResolution.QUARTER_HOUR: 'QUARTER_HOUR'>) aggregate_type= bill start_date= 2024-07-16 20:17:21.879052 end_date= 2024-07-23 20:17:21.879069
start_time      end_time        consumption     provided_cost   start_minus_prev_end    end_minus_prev_end

Getting historical data: account= Account(customer=Customer(uuid='11111111-2222-3333-4444-555555555555'), uuid='11111111-2222-3333-4444-555555555555', utility_account_id='123456787654321', id='11111111-2222-3333-4444-555555555555', meter_type=<MeterType.GAS: 'GAS'>, read_resolution=<ReadResolution.HOUR: 'HOUR'>) aggregate_type= bill start_date= 2024-07-16 20:17:21.879052 end_date= 2024-07-23 20:17:21.879069
start_time      end_time        consumption     provided_cost   start_minus_prev_end    end_minus_prev_end

masked out my real account number.

raidenii commented 2 months ago

Interstingly, if not specifying aggregateType=day, then the API endpoint returns monthly data with correct cost. But once specifying that, while value (consumption) is still returned, providedCost is always returned 0. So it's not the opower code here but the API might be malfunctioning... @holocronology , where do you download the daily data? I can only find usage but not daily cost.

tronikos commented 2 months ago

The default value of aggregate_type is day so there should be no difference.

Can you run: python -m opower --aggregate_type bill --start_date=2024-01-01 Do you get cost data?

raidenii commented 2 months ago

Yes, up to the last bill.

start_time      end_time        consumption     provided_cost   start_minus_prev_end    end_minus_prev_end
2024-04-17 00:00:00-04:00       2024-04-27 00:00:00-04:00       19.0    13.22   None    None
2024-04-27 00:00:00-04:00       2024-05-29 00:00:00-04:00       60.0    38.63   0:00:00 32 days, 0:00:00
2024-05-29 00:00:00-04:00       2024-06-27 00:00:00-04:00       217.0   87.96   0:00:00 29 days, 0:00:00
start_time      end_time        consumption     provided_cost   start_minus_prev_end    end_minus_prev_end
2024-04-17 00:00:00-04:00       2024-04-27 00:00:00-04:00       2.0     13.71   None    None
2024-04-27 00:00:00-04:00       2024-05-29 00:00:00-04:00       3.0     35.84   0:00:00 32 days, 0:00:00
2024-05-29 00:00:00-04:00       2024-06-27 00:00:00-04:00       3.0     32.67   0:00:00 29 days, 0:00:00

The default value of aggregate_type is day so there should be no difference.

Sorry, I didn't make this clear. My statement of

if not specifying aggregateType=day

was regarding that I was using Postman (instead of opower module) to test. I believe the default behavior of the API endpoint is aggregateType=bill.

raidenii commented 2 months ago

Weird, the gas consumption sensor suddenly disappeared yesterday, and reinstallation of the integration did not help. After deleting the records from statistics data, the integration won't even authenticate at all, keep saying 2FA secret is wrong (while the exact same secret works fine for python module).

holocronology commented 2 months ago

Weird, the gas consumption sensor suddenly disappeared yesterday, and reinstallation of the integration did not help. After deleting the records from statistics data, the integration won't even authenticate at all, keep saying 2FA secret is wrong (while the exact same secret works fine for python module).

Not dissimilar to the issue I had above with the 2FA secret.

holocronology commented 2 months ago

Any updates on this?

raidenii commented 2 months ago

My guess this is unsolvable since there is no cost information (for hourly/quarter hour usage) provided by the API, or at least that's the case for me. You can pull the data from API and verify yourself.

My gas sensors are missing, also due to the API not providing the forecast. ConEd API seems broken at this moment for me.

holocronology commented 2 months ago

I would still like the option to manually enter the cost per kWh or gas as an override for these sorts of cases. An estimate is better than nothing.

raidenii commented 2 months ago

I think it's possible to build a number helper with the unit price, then the energy dashboard can use that to calculate the price.

It will require a bit of work to determine the rate based date and usage (which can be extracted from statistics, I think), though (if on standard delivery plan). Time-of-use can be similar too.

holocronology commented 2 months ago

I think it's possible to build a number helper with the unit price, then the energy dashboard can use that to calculate the price.

It will require a bit of work to determine the rate based date and usage (which can be extracted from statistics, I think), though (if on standard delivery plan). Time-of-use can be similar too.

Why are these two options not selectable? image

It seems that the ability to manually enter cost data exists.

devbyaccident commented 3 weeks ago

Hey folks, confirming this is also an issue for PECO utilities. Would be fine if I could set a static price or an entity with the current price, not sure why those are locked out when consumed energy is set to use the OPower integration.

holocronology commented 3 weeks ago

I mean, why are those options even there if you can't use them?