0.8.0 Refactoring code - Githubissues

emptywee commented 6 months ago

Refactor code
Clean up
Make code more pythonish

@jasonacox I know it's a little too much to review, but what do you think if we go this way as the first round of refactoring? It's a little hard to sync it with upstream changes, but if we transition to this structure, it'll be easy to both continue maintaining the current code and refactoring it further.

I noticed that meaning of the jsonformat param was inconsistent across the board, so I fixed that as well, but it might break things for any users of the library if they used the methods directly. Mainly it was the poll() method, which treated the param the other way around.

I also asked about code style guide or anything along those lines, you didn't see it I guess :) So I made it look a bit more Python-ish, if you don't mind.

Please, let me know what you think.

emptywee commented 6 months ago

Oh, oops, didn't know example.py was required for tests :) I basically renamed it to example-local-mode.py, let me rename it back.

jasonacox commented 6 months ago

🤯 !! You did this today?! Amazing @emptywee ! A lot to review but I like the style consistency and the jsonformat change.

So far, I like what I see. As you mention, we should look at the breaking changes and call them out. Also, proxy is not running. I will probably start with that and back in to what is breaking (first pass looks like it is related to missing pw.auth['AuthCookie'].

emptywee commented 6 months ago

!! You did this today?! Amazing @emptywee ! A lot to review but I like the style consistency and the jsonformat change.

Yeah :) It's not much, tbh, just moved some code around, so that we can work on it further later.

So far, I like what I see. As you mention, we should look at the breaking changes and call them out. Also, proxy is not running. I will probably start with that and back in to what is breaking (first pass looks like it is related to missing pw.auth['AuthCookie'].

Oopsie, I am not super familiar with the project and all moving parts :( I'll look into the proxy piece, too.

emptywee commented 6 months ago

Fixed the proxy piece to be compatible with the new structure of the library. Tested on my local - worked well with both local powerwall and cloud access.

emptywee commented 6 months ago

Also, I saw some kind of an effort to support py2? I think we can remove it unless it's required. We don't mention it on pypi, we don't run any tests with it either. If we have to support py2, I'll have to drop all the type hints I've added with this PR.

@jasonacox what do you think?

jasonacox commented 6 months ago

I saw some kind of an effort to support py2?

Yes, we can remove py2 support. That's just legacy code. We also assert Python 3 in our setup.py https://github.com/jasonacox/pypowerwall/blob/7b1fc457d4124727b555cdf05b4fcab4ec412750/setup.py#L24.

emptywee commented 6 months ago

Great, I'll remove it then as part of this code refresh. Thanks.

emptywee commented 6 months ago

Removed py2 stuff, looks like it was only one script there.

emptywee commented 6 months ago

@jasonacox what do you think about 1ccfbc3598617615e3932f0e8aa3f4a1f8190afb ? If we get rid of tens of if-elif-elif-elif in the poll function there? Not very cryptic, I hope :)

jasonacox commented 6 months ago

I am doing casual testing on this @emptywee - but let me know if you think this is ready to shift out of "(WIP)" mode to real testing for merge.

emptywee commented 6 months ago

@jasonacox I believe it is. I thought we were waiting for the dude you asked to four eye this PR.

I wouldn't want to make any other changes right now, since it would make it even harder to review.

You have any means to test it thoroughly? Should we start covering the library with unit tests?

Also, the longer we wait, the harder it is to keep up with other proposed or planned changes.

On Thu, Mar 21, 2024, 12:19 AM Jason Cox @.***> wrote:

I am doing casual testing on this @emptywee https://github.com/emptywee

but let me know if you think this is ready to shift out of "(WIP)" mode to real testing for merge.

— Reply to this email directly, view it on GitHub https://github.com/jasonacox/pypowerwall/pull/77#issuecomment-2011239428, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABGEAPC6Z2OGQEC2FDUQHYLYZJUWPAVCNFSM6AAAAABE2WKVMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJRGIZTSNBSHA . You are receiving this because you were mentioned.Message ID: @.***>

mcbirse commented 6 months ago

I thought we were waiting for the dude you asked to four eye this PR.

The Dude will try to review/test this weekend. Cheers.

emptywee commented 6 months ago

I thought we were waiting for the dude you asked to four eye this PR.

The Dude will try to review/test this weekend. Cheers.

Ahahah, I forgot the name replying via email. Sorry!

Thanks for taking a look at it during the weekend, cheers!

jasonacox commented 6 months ago

@mcbirse you literally had me 🤣 Thank you! 🙏

@emptywee Thanks again for all of this. We are in a deadlock waiting for each other! 😂 I

More than unit test, I would love to add a real test to the CI. I built the pwsimulator (and container) for that purpose but never had time to build the action. It even includes the old vitals protobuf binary simulation. I may try to get it working...

But for this, I also want to test across multiple architectures (MacOS, Linux, WinOS, RPi) and setups. Thankfully, the proxy is good at exercising the API.

“The Dude abides.”

emptywee commented 6 months ago

@jasonacox lol it cracked me up too :D Anyway, I am pretty sure 99-100% should work as it used to, and even better =)

Let me know what I can do to help with testing/merging it sooner than later!

jasonacox commented 6 months ago

Test Results

Platforms

✅ MacOS ✅ RPi ✅ Linux (Ubuntu) ✅ Win11

Modes

✅ Local ❌ Cloud - Error with setup

Proxy Test

The proxy docker container is available at jasonacox/pypowerwall:0.8.0t51 tested with Powerwall-Dashboard ✅ MacOS ✅ RPi ✅ Linux (Ubuntu) ✅ Win11

mcbirse commented 6 months ago

I have just started testing, and noting the below error/exception is continually logged for me (with or without debug mode).

Running pypowerwall proxy and full dashboard setup w/ telegraf etc.

Calls to "/strings" will always return an exception. We may want to look at suppressing or altering the response when strings do not exist for a user, so the exception is not logged? (I do not have a PW+, just plain PW2 with non-Tesla inverters).

03/24/2024 04:18:24 PM [proxy] [DEBUG] 127.0.0.1 "GET /strings HTTP/1.1" 200 -
03/24/2024 04:18:24 PM [pypowerwall.local.pypowerwall_local] [ERROR] Unable to parse payload as JSON: Expecting value: line 1 column 1 (char 0)

EDIT: Note I was upgraded to 24.4.0 on Friday afternoon....

03/24/2024 04:26:25 PM [pypowerwall.local.pypowerwall_local] [ERROR] 404 Powerwall API not found at https://192.168.222.2/api/devices/vitals
03/24/2024 04:26:25 PM [pypowerwall.local.pypowerwall_local] [ERROR] Firmware 240400 detected - Does not support vitals API - disabling.

mcbirse commented 6 months ago

@mcbirse you literally had me 🤣 Thank you! 🙏

Glad I could give you all a bit of a laugh, lol.

“The Dude abides.”

🤣

jasonacox commented 6 months ago

I spotted a setup issue (on Win11 but likely not alone) when you run python3 -m pypowerwall setup -email=xxx

Error: pyPowerwall [0.8.0] - Cloud Mode Setup

Missing auth file .auth/.pypowerwall.auth - run setup Traceback (most recent call last): File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/app/pypowerwall/main.py", line 66, in c = PyPowerwallCloud(None, authpath=authpath) File "/app/pypowerwall/cloud/pypowerwall_cloud.py", line 86, in init raise PyPowerwallCloudNoTeslaAuthFile(msg) pypowerwall.cloud.exceptions.PyPowerwallCloudNoTeslaAuthFile: Missing auth file .auth/.pypowerwall.auth - run setup

jasonacox commented 6 months ago

Calls to "/strings" will always return an exception.

Good find!! I have strings so it wasn't showing up. We need to catch and handle that condition (no response) instead of erroring.

emptywee commented 6 months ago

Ugh, weird. Do you want me to look into those two issues with /strings and win11 setup call? I don't have win11 env here, but I can try to reproduce it once I manage to set up one.

mcbirse commented 6 months ago

This is the new code section causing the exception to be raised for me:

            if raw:
                payload = r.raw.data
            else:
                payload = r.text
                try:
                    payload = json.loads(payload)
                except Exception as exc:
                    log.error(f'Unable to parse payload as JSON: {exc}')
                    return None
            self.pwcache[api] = payload
            self.pwcachetime[api] = time.perf_counter()
            return payload

vs. old code

            if(raw):
                payload = r.raw.data
            else:
                payload = r.text
            self.pwcache[api] = payload
            self.pwcachetime[api] = time.perf_counter()
        if(jsonformat):
            try:
                data = json.loads(payload)
                return data
            except:
                log.debug('ERROR invalid json response: %r' % payload)
                return None
        else:
            return payload

The old code checks the format of the payload before trying json.loads(payload). Probably my payload response has always been something else (null maybe? will check) and it just returned whatever the payload contained, and never raised/logged any error.

emptywee commented 6 months ago

@mcbirse yeah, possible. We can easily fix it. This is in part because jsonformat had a different meaning there.

mcbirse commented 6 months ago

Just noting also, pypowerwall proxy 0.7.12 in DEBUG mode outputs below error for request to "/strings" for me.

03/24/2024 04:57:39 PM [pypowerwall] [DEBUG] pypowerwall [0.7.12]
03/24/2024 04:57:39 PM [proxy] [INFO] pyPowerwall Proxy Started
03/24/2024 04:57:39 PM [pypowerwall] [DEBUG] Tesla local mode enabled
03/24/2024 04:57:48 PM [pypowerwall] [DEBUG] ERROR invalid json response: ''

But this is only logged in DEBUG mode though, so normally would never see an error in the logs. Either way for users without strings I think an error should be suppressed.

emptywee commented 6 months ago

Just noting also, pypowerwall proxy 0.7.12 in DEBUG mode outputs below error for request to "/strings" for me.
03/24/2024 04:57:39 PM [pypowerwall] [DEBUG] pypowerwall [0.7.12]
03/24/2024 04:57:39 PM [proxy] [INFO] pyPowerwall Proxy Started
03/24/2024 04:57:39 PM [pypowerwall] [DEBUG] Tesla local mode enabled
03/24/2024 04:57:48 PM [pypowerwall] [DEBUG] ERROR invalid json response: ''
But this is only logged in DEBUG mode though, so normally would never see an error in the logs. Either way for users without strings I think an error should be suppressed.

Agreed. We can do it.

jasonacox commented 6 months ago

Ugh, weird. Do you want me to look into those two issues with /strings and win11 setup call? I don't have win11 env here, but I can try to reproduce it once I manage to set up one.

It happens on all platforms. It just happens that I had never set up Cloud mode on this Win11 test host so the auth file didn't exist

emptywee commented 6 months ago

Ugh, weird. Do you want me to look into those two issues with /strings and win11 setup call? I don't have win11 env here, but I can try to reproduce it once I manage to set up one.

It happens on all platforms. It just happens that I had never set up Cloud mode on this Win11 test host so the auth file didn't exist

Ok, I'll check. It's a little late here in Texas, I'll check tomorrow.

emptywee commented 6 months ago

Ok, I am on it now.

@jasonacox any reason we don't use the standard argparse module of python there for setup, scan, and other optional parameters?

jasonacox commented 6 months ago

@jasonacox any reason we don't use the standard argparse module of python there for setup, scan, and other optional parameters?

Nope, that was just me not know about argparse when I built this. I'm good with changing to that but we need to ensure that all CLI arguments work the same a they do now for dependent projects / scripts (ie. don't switch from scan to --scan).

emptywee commented 6 months ago

Nope, that was just me not know about argparse when I built this. I'm good with changing to that but we need to ensure that all CLI arguments work the same a they do now for dependent projects / scripts (ie. don't switch from scan to --scan).

We can definitely leave setup/scan as is, the -email might need to be changed to --email though, I will have to look if you can change the way it parses args from command line.

emptywee commented 6 months ago

@mcbirse Michael, the /strings should be fixed now and stop spamming the json error message every time the strings url is requested from the proxy. May need a fresh build of the proxy piece though :)

jasonacox commented 6 months ago

Thanks @emptywee 🙏 - I'll build a new container to test.

We can definitely leave setup/scan as is, the -email might need to be changed to --email though, I will have to look if you can change the way it parses args from command line.

We can also update it in a future PR. IMHO, it isn't critical for this foundational refactoring. That's particularly true if we make a breaking change to the interface (e.g. --email vs -email). Instead of adding more scope, I think it would be good to focus on just solving the issues found in testing so we can complete the merge.

emptywee commented 6 months ago

We can also update it in a future PR. IMHO, it isn't critical for this foundational refactoring. That's particularly true if we make a breaking change to the i

Yeah, but it looks like quite a simple change :) The only problem is the timeout positional argument, which is kinda optional in one case, but kinda required in the other, based on the original usage information printed =) Despite having a default value. If we can make it to be an optional arg such as -timeout= too, the transition to argparse will be trivial.

emptywee commented 6 months ago

I take it back, we can leave the timeout positional argument as is. I am testing it right now.

emptywee commented 6 months ago

@jasonacox see https://github.com/jasonacox/pypowerwall/pull/77/commits/de0eb3dd92ac863aa0accf4f2e57234b4a6944f3 - this mirrors the usage. The only conflicting thing for me is the timeout positional argument, which also has a default value, which makes it optional and should be moved to -timeout, in my opinion. Because right now argparse makes it mandatory, i.e. its value to be provided if scan is chosen. I suggest we move it to optional -timeout argument and update documentation/scripts where it might be in use.

emptywee commented 6 months ago

@jasonacox see https://github.com/jasonacox/pypowerwall/pull/77/commits/9251d0c8b87a979d3c3e141c94f461deea852459 what I propose. If disagree, I'll revert.

jasonacox commented 6 months ago

I tested the command line in/outside of docker and seem to work with all the test cases for setup and scan I can think of.

New container: jasonacox/pypowerwall:0.8.0t51-beta

However, there is still something wrong with our AuthFile logic.

$ docker exec -it pypowerwall python3 -m pypowerwall setup -email=jason@example.com
pyPowerwall [0.8.0] - Cloud Mode Setup

Tesla Account Setup
------------------------------------------------------------

  Email address: jason@example.com

Open the below address in your browser to login.

https://auth.tesla.com/oauth2/v3/authorize?response_type=code&client_id=ownerapi&redirect_uri=https%3A%2F%2Fauth.tesla.com%....

After login, paste the URL of the 'Page Not Found' webpage below.

Enter URL after login: https://auth.tesla.com/void/callback?code=....
------------------------------------------------------------
Missing auth file .pypowerwall.auth - run setup
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/app/pypowerwall/__main__.py", line 64, in <module>
    if c.setup(email):
  File "/app/pypowerwall/cloud/pypowerwall_cloud.py", line 857, in setup
    if not self.connect():
  File "/app/pypowerwall/cloud/pypowerwall_cloud.py", line 138, in connect
    raise PyPowerwallCloudNoTeslaAuthFile(msg)
pypowerwall.cloud.exceptions.PyPowerwallCloudNoTeslaAuthFile: Missing auth file .pypowerwall.auth - run setup

The environment (docker exec -it pypowerwall sh) is getting the right path (.auth is the directory it should use):

PWD='/app'
PW_AUTH_PATH='.auth'

emptywee commented 6 months ago

@jasonacox oddly enough I could not reproduce it on my end, but try with the last commit in effect?

emptywee commented 6 months ago

This is what I was getting:

$ docker run -ti -e PW_AUTH_PATH=".auth" --entrypoint=/bin/sh jasonacox/pypowerwall:0.8.0t51-beta
/app # python3 -m pypowerwall setup -email=xyz@xyz.com
pyPowerwall [0.8.0] - Cloud Mode Setup

Tesla Account Setup
------------------------------------------------------------

  Email address: xyz@xyz.com

Open the below address in your browser to login.

https://auth.tesla.com/oauth2/v3/authorize?response_type=code...

After login, paste the URL of the 'Page Not Found' webpage below.

Enter URL after login: https://auth.tesla.com/void/callback?code=NA_...
------------------------------------------------------------

1 Sites Found (* = default)
------------------------------------------------------------
 *1 - Home  (xyz) - Type: battery

jasonacox commented 6 months ago

Doing my final testing. @mcbirse if you have time, check to see if jasonacox/pypowerwall:0.8.0t51-beta2 solves the string errors.

jasonacox commented 6 months ago

Do we really need this or perhaps move this to DEBUG? This is going to be too noisy for cloud mode users of the proxy.

03/24/2024 02:40:33 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:33 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:33 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:33 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:33 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:34 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:34 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:34 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:36 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:36 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.

emptywee commented 6 months ago

Do we really need this or perhaps move this to DEBUG? This is going to be too noisy for cloud mode users of the proxy.

03/24/2024 02:40:33 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:33 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:33 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:33 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:33 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:34 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:34 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:34 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:36 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:40:36 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.

Hmm, what if we print it once and suppress further?

emptywee commented 6 months ago

Why do we feed proxy users in cloud mode with mock data, btw?

jasonacox commented 6 months ago

Hmm, what if we print it once and suppress further?

Actually, I think it would be better if it was included in the DEBUG line with the API information. Here is a snapshot of logs with a basic dashboard running (power flow animation):

03/24/2024 02:50:35 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Returning cached SITE_DATA data
03/24/2024 02:50:35 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Request for /api/system_status/grid_status
03/24/2024 02:50:35 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Returning cached SITE_DATA data
03/24/2024 02:50:35 PM [proxy] [DEBUG] 172.18.0.4 "GET /soe HTTP/1.1" 200 -
03/24/2024 02:50:35 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Request for /api/system_status/soe
03/24/2024 02:50:35 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Returning cached SITE_SUMMARY data
03/24/2024 02:50:35 PM [proxy] [DEBUG] 172.18.0.4 "GET /temps/pw HTTP/1.1" 200 -
03/24/2024 02:50:35 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Request for /vitals
03/24/2024 02:50:35 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Returning cached SITE_CONFIG data
03/24/2024 02:50:35 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Returning cached SITE_DATA data
03/24/2024 02:50:35 PM [proxy] [DEBUG] 172.18.0.4 "GET /strings HTTP/1.1" 200 -
03/24/2024 02:50:35 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Request for /vitals
03/24/2024 02:50:35 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Returning cached SITE_CONFIG data
03/24/2024 02:50:35 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Returning cached SITE_DATA data
03/24/2024 02:50:36 PM [proxy] [DEBUG] 192.168.65.1 "GET /api/system_status/grid_status HTTP/1.1" 200 -
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Request for /api/system_status/grid_status
03/24/2024 02:50:36 PM [proxy] [DEBUG] 192.168.65.1 "GET /api/sitemaster HTTP/1.1" 200 -
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Request for /api/sitemaster
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:50:36 PM [proxy] [DEBUG] 192.168.65.1 "GET /api/meters/aggregates HTTP/1.1" 200 -
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Request for /api/meters/aggregates
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Returning cached SITE_CONFIG data
03/24/2024 02:50:36 PM [proxy] [DEBUG] 192.168.65.1 "GET /api/system_status/soe HTTP/1.1" 200 -
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Request for /api/system_status/soe
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Returning cached SITE_SUMMARY data
03/24/2024 02:50:36 PM [proxy] [DEBUG] 192.168.65.1 "GET /api/powerwalls HTTP/1.1" 200 -
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Request for /api/powerwalls
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Retrieved SITE_DATA data
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Returning cached SITE_DATA data

We could just decorate the DEBUG line with "[mock data]" qualifier and turn

2:50:36 PM [proxy] [DEBUG] 192.168.65.1 "GET /api/sitemaster HTTP/1.1" 200 -
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Request for /api/sitemaster
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API is using mock data in cloud mode.

into

2:50:36 PM [proxy] [DEBUG] 192.168.65.1 "GET /api/sitemaster HTTP/1.1" 200 -
03/24/2024 02:50:36 PM [pypowerwall.cloud.pypowerwall_cloud] [DEBUG]  -- cloud: Request for /api/sitemaster [mock data]

Why do we feed proxy users in cloud mode with mock data, btw?

The intent of cloud mode is to simulate local mode as a drop-in replacement. Tools built for local mode access to powerwall can use the same API even if their system doesn't have a local API (e.g. Powerwall 3).

emptywee commented 6 months ago

@jasonacox
How about https://github.com/jasonacox/pypowerwall/pull/77/commits/bf5f6ef75d3557a8a26349b70666315e8664f1a3 ?

jasonacox commented 6 months ago

That works! Proxy logs now using jasonacox/pypowerwall:0.8.0t51-beta3:

03/24/2024 03:31:24 PM [proxy] [INFO] pyPowerwall [0.8.0] Proxy Server [t51] - HTTP Port 8675
03/24/2024 03:31:24 PM [proxy] [INFO] pyPowerwall Proxy Started
03/24/2024 03:31:24 PM [proxy] [INFO] pyPowerwall Proxy Server - Cloud Mode
03/24/2024 03:31:25 PM [proxy] [INFO] Connected to Site ID 35...9 (Tesla Energy Gateway)
03/24/2024 03:31:32 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API [get_api_sitemaster] is using mock data in cloud mode. This message will be printed only once at the warning level.
03/24/2024 03:31:32 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API [get_api_powerwalls] is using mock data in cloud mode. This message will be printed only once at the warning level.
03/24/2024 03:31:32 PM [pypowerwall.cloud.pypowerwall_cloud] [WARNING] This API [get_api_auth_toggle_supported] is using mock data in cloud mode. This message will be printed only once at the warning level.

Tested on: ✅ Win11 ✅ MacOS ✅ RPi ✅ Ubuntu Linux

jasonacox commented 6 months ago

New proxy container built with latest commit: jasonacox/pypowerwall:0.8.0t51-beta4

Functions tested (local and cloud mode):

import pypowerwall

pw = pypowerwall.Powerwall(host,password,email,timezone)

pw.poll()
pw.level()
pw.power()
pw.site()
pw.solar()
pw.battery()
pw.load()
pw.grid()
pw.home()
pw.vitals()
pw.strings()
pw.din()
pw.uptime()
pw.version()
pw.status()
pw.site_name()
pw.temps()
pw.alerts()
pw.system_status()
pw.battery_blocks()
pw.grid_status()
pw.is_connected()
pw.get_reserve()
pw.get_time_remaining()

emptywee commented 6 months ago

Kind like how it's working out :)

jasonacox commented 6 months ago

Thanks again for this @emptywee ! - I'm ready to merge unless either of you spot anything.

@mcbirse if you have time, check to see if jasonacox/pypowerwall:0.8.0t51-beta4 solves the string errors. I'll merge tomorrow if all goes well.

emptywee commented 6 months ago

Thanks again for this @emptywee ! - I'm ready to merge unless either of you spot anything.

@mcbirse if you have time, check to see if jasonacox/pypowerwall:0.8.0t51-beta4 solves the string errors. I'll merge tomorrow if all goes well.

Happy to help!

Do you want to keep it as something that only polls data? I've already extended the 0.7.x version of your library with a post method to change battery reserve settings, if you are cool with it, I'll open a PR when I adapt it for 0.8.0. Unfortunately, local mode doesn't allow you to control battery reserve threshold anymore (starting from some firmware), but it is what it is.

jasonacox / pypowerwall

0.8.0 Refactoring code #77

Test Results

Platforms

Modes

Proxy Test