search

zabuldon / teslajsonpy

Apache License 2.0
56 stars 62 forks source link

"grid_status" and "solar_power" values #334

Closed shred86 closed 2 years ago

shred86 commented 2 years ago

This is minor but I wanted to document this here to maybe fix in the future. I noticed one of the comments in power.py mentions:

            # Note: Some systems that pre-date Tesla aquisition of SolarCity will have `grid_status: Unknown`,
            # but will have solar power values. At the same time, newer systems will report spurious reads of 0 Watts
            # and grid status unknown. If solar power is 0 return null.

I have a Tesla solar system with a Tesla inverter (no powerwalls) + Neurio and grid_status is always Unknown. As such, receiving a solar_power value of 0 with grid_status of Unknown is actually not a spurious energy site power read as currently being logged.

We probably need to get JSON responses from the different Tesla solar configurations and update the unofficial Tesla API documentation. I've already added the responses for my setup, but there's obviously a wide range of possible setups. These are the various components I'm aware of, and I'm sure I missed a couple. However, I think how the data is reported is largely going to be based on the gateway.

PV arrays:

Inverters:

Grid monitoring:

Batteries:

Gateways:

purcell-lab commented 2 years ago

I have SolarEdge with Powerwall, is there a specific script I could run to get the relevant JSON response?

shred86 commented 2 years ago

I have SolarEdge with Powerwall, is there a specific script I could run to get the relevant JSON response?

Edit: See post below for easier solution to get live_status JSON response.

You can use TeslaPy's CLI tool but it is missing some arguments for the endpoints we want, so I forked it and added them. I haven't submitted a pull request yet because this is just a temporary solution but you can just download them from my forked repo. If you're on Mac OS, I was running into issues where I couldn't type in my password in the login screen. I ended up using Chrome but that requires some additional code changes and installation of ChromeDriver which I can highlight in a separate post if it becomes an issue.

Apologize if you already know this but once you download it, you'll have to navigate to the directory from the terminal and:

pip install -r requirements to download the libraries that TeslaPy uses. python3 cli.py -e yourname@email.com --site to get current site data python3 cli.py -e yourname@email.com --site_config to get site config data python3 cli.py -e yourname@email.com --site_summary to get site summary data

The first time you use the CLI tool, it will open a window to the Tesla login screen. Your email address should already be populated so just type in your password and login.

alandtse commented 2 years ago

Alternatively, if you enable teslajsonpy debugging in the HA component, all API calls are logged

shred86 commented 2 years ago

That...is so much easier, lol. Although we're only using the live_status endpoint (aka site or SITE_DATA), so to get the other two you'll have to use TeslaPy (for populating the unofficial API documentation).

purcell-lab commented 2 years ago

Alternatively, if you enable teslajsonpy debugging in the HA component, all API calls are logged

Unfortunately I'm still not seeing my battery via HA, with your recent update, anything I should do to enable?

purcell-lab commented 2 years ago

pip install -r requirements to download the libraries that TeslaPy uses. python3 cli.py -e yourname@email.com --site to get current site data python3 cli.py -e yourname@email.com --site_config to get site config data python3 cli.py -e yourname@email.com --site_summary to get site summary data

Thanks using cli.py with -b option shows my powerwall/ battery details, when I use your cli.py from your fork I don't get anything returned for your --site options?

shred86 commented 2 years ago

Alternatively, if you enable teslajsonpy debugging in the HA component, all API calls are logged

Unfortunately I'm still not seeing my battery via HA, with your recent update, anything I should do to enable?

There won’t be any battery data in Home Assistant itself yet as it hasn’t been implemented. However, posting the debug data for the JSON response when it update every 10 sec might still be useful to look at.

pip install -r requirements to download the libraries that TeslaPy uses. python3 cli.py -e yourname@email.com --site to get current site data python3 cli.py -e yourname@email.com --site_config to get site config data python3 cli.py -e yourname@email.com --site_summary to get site summary data

Thanks using cli.py with -b option shows my powerwall/ battery details, when I use your cli.py from your fork I don't get anything returned for your --site options?

Interesting, I suspect whichever gateway you have is probably reporting all info through the battery/ endpoints only then. Do you get even an empty list or anything when using the —site, —site_config or —site_summary options? If you could post what you get using -l and -b, that would be helpful to look at. Remove any “personal” info like addresses or data that’s unique to your setup (e.g. IDs). I can update the CLI tool tomorrow morning so you can try some of the other battery endpoints.

shred86 commented 2 years ago

@purcell-lab I updated my TeslaPy repo but the only thing I added was the CLI argument --battery_summary as that's the only other GET request I see that's battery related. If you don't mind trying the following and reporting which each one shows. I'd be curious if you're getting absolutely nothing back, an empty list/dictionary, etc.

Thanks for helping with this. I'm sure you're aware the Tesla Powerwall HA integration should expose everything to HA for powerwall users, and it's local.

Also, can you confirm which gateway you have? I'm only a little familiar with Tesla solar systems with powerwalls and how they're setup, but if you have a SolarEdge inverter, my understanding was you would also have a separate gateway which is actually what connects to the internet to pass data to the Tesla servers. Does your powerwall connect to the internet too?

Edit: Just updated my TeslaPy fork. I noticed an issue that I fixed and added a -z argument. This is basically the same as the API option -a but this will work for battery and energy sites that require the id or energy_site_id in the URI.

purcell-lab commented 2 years ago

@shred86 thanks for your work on this.

I am using the Tesla Powerwall local API integration and that works well, but for control one needs to access the cloud API. image

I am also using https://github.com/carboncoop/tesla-gateway-ha-component, which needs some love and I have an open PR to make that functional, but I think this path to integration into HA is looking more mature (some of the JSON response here): https://github.com/alandtse/tesla/issues/79#issuecomment-1109224461

My setup, I have 2x 7kW SolarEdge 3 phase inverters and a single Powerwall2 with a home energy gateway, I think the HEG connects to the Internet, not the Powerwall direct.

Screen recording 2022-08-16 15.41.13.webm

$ python3 ./cli.py.1 -e mark@purcell.id.au --list
2022-08-16 15:29:33,084 - root - INFO - 2 product(s), 2 selected
Product 0:
{
    "id": XXX,
    "vehicle_id": XXX,
    "vin": "XXX",
    "display_name": "Duka",
    "option_codes": "AD15,AF00,APFB,APH4,AU3P,BC3R,BT37,CDM0,CH06,COAU,DRRH,DV4W,FC3P,FG31,FM3P,GLFR,HL31,HM30,ID3W,IL31,LTPB,MDL3,MR30,PMNG,PC31,REAP,RF3G,RS3H,S3PB,SA3P,SC04,STCP,SU3C,T3M3,TM00,TW00,UT3P,W32D,WR00,ZINV,MI01,PL31,SLR1,ST30,BG32,I38M,OSSB,AUF1,RSF1,ILF1,FGF1,CPF1,HP30,PT00",
    "color": null,
    "access_type": "OWNER",
    "tokens": [
        "XXX",
        "XXX"
    ],
    "state": "online",
    "in_service": false,
    "id_s": "XXX",
    "calendar_enabled": true,
    "api_version": 42,
    "backseat_token": null,
    "backseat_token_updated_at": null
}
Product 1:
{
    "energy_site_id": XXX,
    "resource_type": "battery",
    "site_name": "Home Energy Gateway",
    "id": "XXX",
    "gateway_id": "XXX",
    "asset_site_id": "XXX",
    "energy_left": 2864.7368421052633,
    "total_pack_energy": 14070,
    "percentage_charged": 20.360603000037408,
    "battery_type": "ac_powerwall",
    "backup_capable": true,
    "battery_power": 3080,
    "storm_mode_enabled": true,
    "powerwall_onboarding_settings_set": true,
    "sync_grid_alert_enabled": true,
    "breaker_alert_enabled": true,
    "components": {
        "battery": true,
        "battery_type": "ac_powerwall",
        "solar": true,
        "solar_type": "pv_panel",
        "grid": true,
        "load_meter": true,
        "market_type": "residential"
    }
}
mark@penguin:~$ python3 ./cli.py.1 -e mark@purcell.id.au --site
2022-08-16 15:30:50,950 - root - INFO - 2 product(s), 2 selected
Product 0:
Product 1:
mark@penguin:~$ python3 ./cli.py.1 -e mark@purcell.id.au --site_config
2022-08-16 15:30:56,908 - root - INFO - 2 product(s), 2 selected
Product 0:
Product 1:
mark@penguin:~$ python3 ./cli.py.1 -e mark@purcell.id.au --site_summary
2022-08-16 15:31:03,695 - root - INFO - 2 product(s), 2 selected
Product 0:
Product 1:
2022-08-16 15:31:12,056 - root - INFO - 2 product(s), 2 selected
Product 0:
Product 1:
{
    "energy_site_id": XXX,
    "resource_type": "battery",
    "site_name": "Home Energy Gateway",
    "id": "XXX",
    "gateway_id": "XXX",
    "asset_site_id": "XXX",
    "energy_left": 12650.052631578948,
    "total_pack_energy": 14069,
    "percentage_charged": 20.360603000037408,
    "battery_type": "ac_powerwall",
    "backup_capable": true,
    "battery_power": 3080,
    "storm_mode_enabled": true,
    "powerwall_onboarding_settings_set": true,
    "sync_grid_alert_enabled": true,
    "breaker_alert_enabled": true,
    "components": {
        "solar": true,
        "solar_type": "pv_panel",
        "battery": true,
        "grid": true,
        "backup": true,
        "gateway": "teg",
        "load_meter": true,
        "tou_capable": true,
        "storm_mode_capable": true,
        "flex_energy_request_capable": false,
        "car_charging_data_supported": false,
        "off_grid_vehicle_charging_reserve_supported": false,
        "vehicle_charging_performance_view_enabled": false,
        "vehicle_charging_solar_offset_view_enabled": false,
        "battery_solar_offset_view_enabled": true,
        "solar_value_enabled": true,
        "energy_value_header": "Energy Value",
        "energy_value_subheader": "Estimated Value",
        "show_grid_import_battery_source_cards": true,
        "backup_time_remaining_enabled": true,
        "rate_plan_manager_supported": true,
        "battery_type": "ac_powerwall",
        "configurable": false,
        "grid_services_enabled": false,
        "customer_preferred_export_rule": "battery_ok",
        "net_meter_mode": "battery_ok"
    },
    "grid_status": "Active",
    "backup": {
        "backup_reserve_percent": 100,
        "events": [
            {
                "timestamp": "2022-07-12T06:56:55+10:00",
                "duration": 38773
            },
            {
                "timestamp": "2022-07-11T20:46:25+10:00",
                "duration": 66479
            },
            {
                "timestamp": "2022-06-29T11:35:43+10:00",
                "duration": 842030
            },
            {
                "timestamp": "2022-06-18T15:28:35+10:00",
                "duration": 1013486
            },
            {
                "timestamp": "2022-06-15T15:43:20+10:00",
                "duration": 210737
            },
            {
                "timestamp": "2022-06-10T08:26:12+10:00",
                "duration": 47649
            },
            {
                "timestamp": "2022-06-03T13:58:52+10:00",
                "duration": 443079
            },
            {
                "timestamp": "2022-05-15T10:46:58+10:00",
                "duration": 31389950
            },
            {
                "timestamp": "2022-05-14T15:33:38+10:00",
                "duration": 1279604
            },
            {
                "timestamp": "2022-05-07T19:39:07+10:00",
                "duration": 901817
            },
            {
                "timestamp": "2022-04-23T08:26:14+10:00",
                "duration": 437693
            },
            {
                "timestamp": "2022-04-22T19:14:33+10:00",
                "duration": 757615
            },
            {
                "timestamp": "2022-04-14T11:54:35+10:00",
                "duration": 581358
            },
            {
                "timestamp": "2022-04-06T22:26:41+10:00",
                "duration": 65188
            },
            {
                "timestamp": "2022-04-03T22:12:07+10:00",
                "duration": 654161
            },
            {
                "timestamp": "2022-04-03T21:57:36+10:00",
                "duration": 798912
            },
            {
                "timestamp": "2022-04-03T18:51:05+10:00",
                "duration": 67764
            },
            {
                "timestamp": "2022-04-03T17:22:58+10:00",
                "duration": 641782
            },
            {
                "timestamp": "2022-04-03T17:21:19+10:00",
                "duration": 69942
            },
            {
                "timestamp": "2022-04-03T06:34:17+10:00",
                "duration": 232350
            },
            {
                "timestamp": "2022-04-02T19:05:41+10:00",
                "duration": 47104
            },
            {
                "timestamp": "2022-04-02T09:35:18+10:00",
                "duration": 258895
            },
            {
                "timestamp": "2022-04-02T05:21:14+10:00",
                "duration": 63814
            },
            {
                "timestamp": "2022-04-01T11:59:57+10:00",
                "duration": 586849
            },
            {
                "timestamp": "2022-04-01T11:50:56+10:00",
                "duration": 457199
            },
            {
                "timestamp": "2022-04-01T11:48:21+10:00",
                "duration": 51065
            },
            {
                "timestamp": "2022-04-01T11:47:23+10:00",
                "duration": 41783
            },
            {
                "timestamp": "2022-04-01T11:01:46+10:00",
                "duration": 73278
            },
            {
                "timestamp": "2022-03-31T17:12:00+10:00",
                "duration": 45838
            },
            {
                "timestamp": "2022-03-24T16:28:07+10:00",
                "duration": 122233
            },
            {
                "timestamp": "2022-03-24T06:15:44+10:00",
                "duration": 5932791
            },
            {
                "timestamp": "2022-03-23T17:01:37+10:00",
                "duration": 210322
            },
            {
                "timestamp": "2022-03-23T16:11:27+10:00",
                "duration": 2608373
            },
            {
                "timestamp": "2022-03-21T21:04:54+10:00",
                "duration": 296080
            }
        ],
        "events_count": 0,
        "total_events": 0
    },
    "user_settings": {
        "storm_mode_enabled": true,
        "powerwall_onboarding_settings_set": true,
        "sync_grid_alert_enabled": false,
        "breaker_alert_enabled": false
    },
    "default_real_mode": "backup",
    "operation": "backup",
    "installation_date": "2022-03-21T17:15:23+10:00",
    "power_reading": [
        {
            "timestamp": "2022-08-16T15:23:24+10:00",
            "load_power": 329,
            "solar_power": 709,
            "grid_power": 2930,
            "battery_power": -3310,
            "generator_power": 0
        }
    ],
    "battery_count": 1
}

You can see above the same power_reading sensors; load_power, solar_power, grid_power as you were talking about so i was hoping these would be exposed into HA.The "operation" sensor is also of great interest, as that is what we change for different modes.

battery_summary doesn't seem to do much for me :-(

$ python3 ./cli.py.1 -e mark@purcell.id.au --battery_summary
2022-08-16 15:37:49,159 - root - INFO - 2 product(s), 2 selected
Product 0:
Product 1:
Traceback (most recent call last):
  File "/home/mark/./cli.py.1", line 221, in <module>
    main()
  File "/home/mark/./cli.py.1", line 91, in main
    print(product.get_battery_summary())
AttributeError: 'Battery' object has no attribute 'get_battery_summary'
shred86 commented 2 years ago

Ah that’s interesting. Just to confirm, using —battery is what got you all the JSON data? Looks like everything is pretty much in that response.

Also, did you happen to download the latest __init__.py file from my forked repo? It looks like you may have just grabbed the latest cli.py? If so, please try downloading __init__.py and trying —battery_summary or better yet, can you try these commands and post the response:

-z BATTERY_DATA -z BATTERY_SUMMARY -z ENERGY_SITE_BACKUP_TIME_REMAINING -z ENERGY_SITE_TELEMETRY_HISTORY

The first one should be the same as before but I think the response might be slightly different based on an issue I found with the CLI tool.

Adding the sensor data should be easy, but I’m hoping we can find someone with the Powerwall+ as well which has the built in Tesla inverter to see what endpoints return data. In the meantime, I can try to start implementing your setup based on the data you’ve provided.

Edit: Do you mind if I post your JSON responses on the Tesla JSON API (Unofficial) site?

shred86 commented 2 years ago

@purcell-lab Just made a new branch and initial commit with what I'm hoping is support for Powerall users to have solar, grid, load and battery power sensor support. Unfortunately I don't have a Powerwall so it's impossible to test except through some limited unit tests.

Since these are changes to the teslajsonpy library, you have to point the Tesla Custom Integration to use my forked version. To do that, open the manifest.json file located in your Home Assistant /config/custom_components/tesla_custom directory and replace this line: "requirements": ["teslajsonpy==2.4.1"],

with this line: "requirements": ["git+https://github.com/shred86/teslajsonpy.git@add-battery-sites#teslajsonpy==3.0.0"],

This is basically saying unless teslajsonpy version 3.0.0 is installed (which we're only on 2.4.1 so that shouldn't be the case for a while), use the forked version.

I also had to uninstall teslajsonpy first to get this to work. For whatever reason, it kept just using the published version. To uninstall teslajsonpy, use pip3 uninstall teslajsonpy.

If we can verify these changes are working, I can try to start incorporating more stuff.

Additional information

There's definitely some differences in how data is published from Tesla depending on your setup. I'm assuming it's based on whichever gateway type you have (e.g. Tesla Inverter vs. Home Energy Gateway which I assume is either X2e, Neo, Series 2). Things I've noticed:

Tesla Inverter "gateway" (non-Powerwall):

Home Energy Gateway (w/ Powerwall 2):

So we basically have to pull the data from different areas depending on your setup. That's all I'm doing right now by using the resource_type key for the JSON response to determine if it's a Tesla Inverter (non-Powerwall) which is solar or a Home Energy Gateway (w/ Powerwall 2) which is battery. The way I'm doing it now works okay but ideally I think it would be better to parse the convert the JSON response into our own dictionary format. That way when we're creating device classes, how you're pulling the data in is consistent and doesn't require the checks I'm doing now. It's not a big deal right now since there's not many different device types for energy sites.

What we really need is some more examples of JSON responses from different setups. I'm really interested in someone with a Powerwall+ that includes the built in Tesla Inverter. The other thing to note is Tesla's API is constantly changing. I've already noticed some new entries in the past couple weeks so I wouldn't be surprised if there's an effort on their end to streamline the API.

purcell-lab commented 2 years ago

@purcell-lab Just made a new branch and initial commit with what I'm hoping is support for Powerall users to have solar, grid, load and battery power sensor support. Unfortunately I don't have a Powerwall so it's impossible to test except through some limited unit tests.

Fantastic, I'm very comfortable to assist with testing/ debugging but won't be able to have a look until tonight.

Thanks for your work on this.

shred86 commented 2 years ago

No worries! I have some updates on issues I already found. I'll push them soon and update this post once it's ready.

Edit: Should be good to go to give it a try.

purcell-lab commented 2 years ago

Edit: Do you mind if I post your JSON responses on the Tesla JSON API (Unofficial) site?

No problem, happy to assist. Happy to uploadmyself as well, if you can point me in the right direction.

-z BATTERY_DATA 

$ python3 ./cli.py.1 -e mark@purcell.id.au -z BATTERY_DATA
2022-08-18 15:11:38,060 - root - INFO - 2 product(s), 2 selected
Product 0:
Product 1:
{
    "response": {
        "site_name": "Home Energy Gateway",
        "energy_left": 0,
        "total_pack_energy": 1,
        "grid_status": "Active",
        "backup": {
            "backup_reserve_percent": 0,
            "events": [
                {
                    "timestamp": "2022-07-12T06:56:55+10:00",
                    "duration": 38773
                },
                {
                    "timestamp": "2022-07-11T20:46:25+10:00",
                    "duration": 66479
                },
                {
                    "timestamp": "2022-06-29T11:35:43+10:00",
                    "duration": 842030
                },
                {
                    "timestamp": "2022-06-18T15:28:35+10:00",
                    "duration": 1013486
                },
                {
                    "timestamp": "2022-06-15T15:43:20+10:00",
                    "duration": 210737
                },
                {
                    "timestamp": "2022-06-10T08:26:12+10:00",
                    "duration": 47649
                },
                {
                    "timestamp": "2022-06-03T13:58:52+10:00",
                    "duration": 443079
                },
                {
                    "timestamp": "2022-05-15T10:46:58+10:00",
                    "duration": 31389950
                },
                {
                    "timestamp": "2022-05-14T15:33:38+10:00",
                    "duration": 1279604
                },
                {
                    "timestamp": "2022-05-07T19:39:07+10:00",
                    "duration": 901817
                },
                {
                    "timestamp": "2022-04-23T08:26:14+10:00",
                    "duration": 437693
                },
                {
                    "timestamp": "2022-04-22T19:14:33+10:00",
                    "duration": 757615
                },
                {
                    "timestamp": "2022-04-14T11:54:35+10:00",
                    "duration": 581358
                },
                {
                    "timestamp": "2022-04-06T22:26:41+10:00",
                    "duration": 65188
                },
                {
                    "timestamp": "2022-04-03T22:12:07+10:00",
                    "duration": 654161
                },
                {
                    "timestamp": "2022-04-03T21:57:36+10:00",
                    "duration": 798912
                },
                {
                    "timestamp": "2022-04-03T18:51:05+10:00",
                    "duration": 67764
                },
                {
                    "timestamp": "2022-04-03T17:22:58+10:00",
                    "duration": 641782
                },
                {
                    "timestamp": "2022-04-03T17:21:19+10:00",
                    "duration": 69942
                },
                {
                    "timestamp": "2022-04-03T06:34:17+10:00",
                    "duration": 232350
                },
                {
                    "timestamp": "2022-04-02T19:05:41+10:00",
                    "duration": 47104
                },
                {
                    "timestamp": "2022-04-02T09:35:18+10:00",
                    "duration": 258895
                },
                {
                    "timestamp": "2022-04-02T05:21:14+10:00",
                    "duration": 63814
                },
                {
                    "timestamp": "2022-04-01T11:59:57+10:00",
                    "duration": 586849
                },
                {
                    "timestamp": "2022-04-01T11:50:56+10:00",
                    "duration": 457199
                },
                {
                    "timestamp": "2022-04-01T11:48:21+10:00",
                    "duration": 51065
                },
                {
                    "timestamp": "2022-04-01T11:47:23+10:00",
                    "duration": 41783
                },
                {
                    "timestamp": "2022-04-01T11:01:46+10:00",
                    "duration": 73278
                },
                {
                    "timestamp": "2022-03-31T17:12:00+10:00",
                    "duration": 45838
                },
                {
                    "timestamp": "2022-03-24T16:28:07+10:00",
                    "duration": 122233
                },
                {
                    "timestamp": "2022-03-24T06:15:44+10:00",
                    "duration": 5932791
                },
                {
                    "timestamp": "2022-03-23T17:01:37+10:00",
                    "duration": 210322
                },
                {
                    "timestamp": "2022-03-23T16:11:27+10:00",
                    "duration": 2608373
                },
                {
                    "timestamp": "2022-03-21T21:04:54+10:00",
                    "duration": 296080
                }
            ],
            "events_count": 0,
            "total_events": 0
        },
        "user_settings": {
            "storm_mode_enabled": true,
            "powerwall_onboarding_settings_set": true,
            "sync_grid_alert_enabled": false,
            "breaker_alert_enabled": false
        },
        "components": {
            "solar": true,
            "solar_type": "pv_panel",
            "battery": true,
            "grid": true,
            "backup": true,
            "gateway": "teg",
            "load_meter": true,
            "tou_capable": true,
            "storm_mode_capable": true,
            "flex_energy_request_capable": false,
            "car_charging_data_supported": false,
            "off_grid_vehicle_charging_reserve_supported": false,
            "vehicle_charging_performance_view_enabled": false,
            "vehicle_charging_solar_offset_view_enabled": false,
            "battery_solar_offset_view_enabled": true,
            "solar_value_enabled": true,
            "energy_value_header": "Energy Value",
            "energy_value_subheader": "Estimated Value",
            "show_grid_import_battery_source_cards": true,
            "backup_time_remaining_enabled": true,
            "rate_plan_manager_supported": true,
            "battery_type": "ac_powerwall",
            "configurable": false,
            "grid_services_enabled": false,
            "customer_preferred_export_rule": "battery_ok",
            "net_meter_mode": "battery_ok"
        },
        "default_real_mode": "self_consumption",
        "operation": "self_consumption",
        "installation_date": "2022-03-21T17:15:23+10:00",
        "power_reading": [
            {
                "timestamp": "2022-08-18T15:10:20+10:00",
                "load_power": 7010,
                "solar_power": 6638,
                "grid_power": 2,
                "battery_power": 370,
                "generator_power": 0
            }
        ],
        "battery_count": 0
    }
}

-z BATTERY_SUMMARY

Interesting total_pack_energy & energy_left are are different to the above.

$ python3 ./cli.py.1 -e mark@purcell.id.au -z BATTERY_SUMMARY
2022-08-18 15:12:24,267 - root - INFO - 2 product(s), 2 selected
Product 0:
Product 1:
{
    "response": {
        "site_name": "Home Energy Gateway",
        "id": "XXX",
        "energy_left": 13610.736842105263,
        "total_pack_energy": 14056,
        "percentage_charged": 96.8322199922116,
        "battery_power": 400
    }
}

-z ENERGY_SITE_BACKUP_TIME_REMAINING

This maybe load_power / energy_left?

But the powerwall cannot discharge at this rate.

$ python3 ./cli.py.1 -e mark@purcell.id.au -z ENERGY_SITE_BACKUP_TIME_REMAINING
2022-08-18 15:12:52,106 - root - INFO - 2 product(s), 2 selected
Product 0:
Product 1:
{
    "response": {
        "time_remaining_hours": 1.97139949280352
    }
}

-z ENERGY_SITE_TELEMETRY_HISTORY

$ python3 ./cli.py.1 -e mark@purcell.id.au -z ENERGY_SITE_TELEMETRY_HISTORY
2022-08-18 15:13:32,904 - root - INFO - 2 product(s), 2 selected
Product 0:
Product 1:
Traceback (most recent call last):
  File "/home/mark/./cli.py.1", line 221, in <module>
    main()
  File "/home/mark/./cli.py.1", line 122, in main
    print(product.api(command, **command_data))
  File "/home/mark/.local/lib/python3.9/site-packages/teslapy/__init__.py", line 726, in api
    return self.tesla.api(name, pathvars, **kwargs)
  File "/home/mark/.local/lib/python3.9/site-packages/teslapy/__init__.py", line 363, in api
    return self.request(endpoint['TYPE'], uri, serialize,
  File "/home/mark/.local/lib/python3.9/site-packages/teslapy/__init__.py", line 155, in request
    response.raise_for_status()  # Raise HTTPError, if one occurred
  File "/home/mark/.local/lib/python3.9/site-packages/requests/models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: `kind` is required for url: https://owner-api.teslamotors.com/api/1/energy_sites/2252097574301386/telemetry_history
shred86 commented 2 years ago

You can submit a pull request at timdorr/tesla-api update that unofficial Tesla API website or I can do it sometime tomorrow. I’d like to format it so we list exactly what our solar setups are to help with building this integration.

As for the responses you’re seeing, only reference the ones you do with the -z command. I noticed that the responses using the arguments as the tool is published was listing out more data then the responses were actually sending and it threw me for a loop for a couple hours. 😆 I opened an issue about it but it’s how the tool was designed. Whenever you request data from an endpoint, it combines it with the PRODUCT_LIST endpoint into a single JSON response. So what you’re seeing isn’t a single JSON response but rather a combination of two. The -z option I added returns only that endpoint response.

This is why you’re seeing different values from using —battery compared to the other day because it combined the response with PRODUCT_LIST which is the —list option. From what I can see, it looks like PRODUCT_LIST and BATTERY_SUMMARY are the two endpoints that provide accurate:

        "energy_left": 13610.736842105263,
        "total_pack_energy": 14056,
        "percentage_charged": 96.8322199922116,
        "battery_power": 400

And BATTERY_DATA provides power_readings:

                "load_power": 7010,
                "solar_power": 6638,
                "grid_power": 2,
                "battery_power": 370,
                "generator_power": 0

It looks like BATTERY_SUMMARY also shows your current battery charge or discharge rate. I’m assuming when it’s discharging, battery_power is a negative number?

Anyways, this is great data and what I implemented so far should work for at least registering the power_readings. I should be able to add the BATTERY_SUMMARY data as well as a sensor. It’s just unfortunate the data is spread across different endpoints. We’re currently polling Tesla servers every 10 seconds to a single endpoint. Not sure if sending requests to two or three every 10 seconds is going to cause issues. I’m working on the rewrite for this integration as well so I’m just going to roll all my changes into there.

purcell-lab commented 2 years ago

I also had to uninstall teslajsonpy first to get this to work. For whatever reason, it kept just using the published version. To uninstall teslajsonpy, use pip3 uninstall teslajsonpy.

Looks like I need to uninstall from within my HA docker instance, which isn't too easy for me at present.

If you increase the version number on your fork does it permit the upgrade?

shred86 commented 2 years ago

Unfortunately it won't because it has to be published to PyPi. I'm going to see if maybe @alandtse can push a new version (or maybe I can create a separate library for testing). At this point, it's probably just worth testing the Tesla Custom Integration re-write after we figure out an easier way to get the updated teslajsonpy library.

alandtse commented 2 years ago

You can manually install any pip library using a git branch including in the ha config directory. Given that, I won't release a test version just for testing. Any releases should be intended for public consumption.

shred86 commented 2 years ago

Yeah that's what we're trying to do but for whatever reason, I have to pip uninstall teslajsonpy when adding a git branch to the manifest.json. Maybe just running this in a dev container would be easiest.

@purcell-lab Do you happen to have VS Code? It's pretty easy to run a dev container, really just open, let it build, and run a task. That way you don't have to mess with your current HA instance either. I've already updated the manifest.json file so you shouldn't have to do anything else.

@alandtse I'm sure it's so obvious I'm just missing it lol, but where is Home Assistant installed in the dev container? Nevermind, found the config directory... it's, /config, 😆

giachello commented 2 years ago

Hey @shred86 I am using the latest versions of tesla_custom and teslajsonpy and am seeing this error now:

2022-08-27 16:09:18.308 ERROR (MainThread) [custom_components.tesla_custom] Unexpected error fetching tesla_custom data: 'grid_status'
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 205, in _async_refresh
    self.data = await self._async_update_data()
  File "/config/custom_components/tesla_custom/__init__.py", line 288, in _async_update_data
    return await self.controller.update()
  File "/usr/local/lib/python3.10/site-packages/teslajsonpy/controller.py", line 1074, in update
    return any(await asyncio.gather(*tasks))
  File "/usr/local/lib/python3.10/site-packages/teslajsonpy/controller.py", line 991, in _get_and_process_energysite_data
    if response["grid_status"] == "Active":
KeyError: 'grid_status'

Will try to collect my data and share. Could you add a DEBUG log where it prints out the response dictionary so we can see exactly what comes out of it?

giachello commented 2 years ago

Hey @shred86
my healthy response is: 

200: {"response":{"solar_power":332,"timestamp":"2022-03-12T17:17:09-08:00"}}

Ie there is no "grid status" when the data read is correct. Can you fix the code to take into account this condition? Thanks!!

Giovanni

shred86 commented 2 years ago

Ahh, that is strange. Here is my response:

{'solar_power': 2640, 'load_power': 8143.0400390625, 'grid_status': 'Unknown', 'grid_services_active': False, 'grid_power': 5503.0400390625, 'grid_services_power': 0, 'generator_power': 0, 'island_status': 'island_status_unknown', 'storm_mode_active': False, 'timestamp': '2022-08-27T16:27:04-07:00', 'wall_connectors': []}

So it only shows the grid_status for you when it's Unknown?

giachello commented 2 years ago

Yes check out the log in this issue:

https://github.com/zabuldon/teslajsonpy/issues/287

giachello commented 2 years ago

In general it's good practice to test for json elements existence before asking for the dict in python because of this type of exception issues.

shred86 commented 2 years ago

Ah, got it. I need to add that to the JSON responses I'm trying to collect. Just to confirm your setup, it's a Solar Edge inverter, no powerwalls and what type of gateway?

Edit: Hm, need to rethink my original implementation as that won't work...

Update: I think this will cover all cases since I'm not sure if there's setups that will report when grid_status is actually Active.

if "grid_status" not in response or response.get("grid_status") != "Unknown":
giachello commented 2 years ago

It's a Delta M inverter, no powerwall. The zigbee gateway is a Neo 2 (black box)

shred86 commented 2 years ago

@giachello Version 2.4.2 of the Tesla Custom Integration is released so hopefully the fix in there resolves the issue.

giachello commented 2 years ago

Ok seems to be working now!