jasonacox / Powerwall-Dashboard

Grafana Monitoring Dashboard for Tesla Solar and Powerwall Systems
MIT License
299 stars 64 forks source link

Local Powerwall not HTTPS #175

Open SomermanM opened 1 year ago

SomermanM commented 1 year ago

When I connect directly to my Powerwall the certificate is not valid (NET::ERR_CERT_AUTHORITY_INVALID) and I have to override the error. This pushes me to port 80. The configuration for pypowerwall uses port 443 and it will not connect. pypowerwall.ConnectionError: Unable to connect to Powerwall at...

Is there a way to force pypowerwall to use port 443?

jasonacox commented 1 year ago

Hi @SomermanM

Keep in mind:

I suspect the error you are getting from pypowerwall is not relate to the cert but is related to connectivity or password settings. Try this from the same host running pypowerwall:

# install pypowerwall module
python3 -m pip install pypowerwall

# scan for powerwall
python3 -m pypowerwall scan

It should discover your powerwall. If it works, you might have used the wrong IP address or password. To test:

import pypowerwall

pw = pypowerwall.Powerwall("10.10.10.10","yourpassword")
print(pw.site_name())
print(pw.status())
SomermanM commented 1 year ago

The ip and password is correct. Now I get this error from the Power Flow panel: [Fiddler] ReadResponse() failed: The server did not return a response for this request.

jasonacox commented 1 year ago

I'm confused. Fiddler is a proxy server. That shouldn't show up in Powerwall-Dashboard. Where are you seeing that? Are you using a web proxy?

SomermanM commented 1 year ago

Here is where I see the message.: image I'm not using a proxy server, but I do have a device that monitors my internet usage, and it might be acting as one. I'll unplug it temporarily and see.

SomermanM commented 1 year ago

image I powered off the meter and rebooted the router and am still getting this message. The only other item of note for my network is that the router is connected to the gateway using a passthrough.

jasonacox commented 1 year ago

Interesting! The weather data showed up after you unplugged the device.

Where are you running the Dashboard services (where you ran setup.sh)? That graph indicates that pypowerwall is not running correctly. Go to http://localhost:8675/stats (replace localhost with the address of the server) and from the command line try this to see if it is running:

# check logs for pypowerwall
docker logs pypowerwall

# also see what is running
docker ps 

It would be good to see what you discover from those.

Seculating, the reason pypowerwall would not run is that it has a bad IP address or password for the Powerwall. Those values are set in the file pypowerwall.env. There are some cases where special symbols in the password will keep it from working as well (see https://github.com/jasonacox/Powerwall-Dashboard/issues/142).

SomermanM commented 1 year ago

I'm running everything on my Windows 11 machine using WSL2 so the URL is localhost. I checked the pypowerwall.env and the password/ip are correct and the pw is letter/number only. I am able to sign in to my powerwall directly using the ip/email/pw image Here is the log: Traceback (most recent call last): 2023-02-11 10:46:21 File "/app/server.py", line 107, in 2023-02-11 10:46:21 pw = pypowerwall.Powerwall(host,password,email,timezone,cache_expire,timeout,pool_maxsize) 2023-02-11 10:46:21 File "/usr/local/lib/python3.10/site-packages/pypowerwall/init.py", line 138, in init 2023-02-11 10:46:21 self._get_session() 2023-02-11 10:46:21 File "/usr/local/lib/python3.10/site-packages/pypowerwall/init.py", line 151, in _get_session 2023-02-11 10:46:21 raise ConnectionError(err) 2023-02-11 10:46:21 pypowerwall.ConnectionError: Unable to connect to Powerwall at https://192.168.18.71 2023-02-11 06:03:40 urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='192.168.18.71', port=443): Read timed out. (read timeout=10)

SomermanM commented 1 year ago

PING 192.168.18.71 (192.168.18.71) 56(84) bytes of data. 64 bytes from 192.168.18.71: icmp_seq=1 ttl=63 time=106 ms 64 bytes from 192.168.18.71: icmp_seq=2 ttl=63 time=15.7 ms 64 bytes from 192.168.18.71: icmp_seq=3 ttl=63 time=37.7 ms 64 bytes from 192.168.18.71: icmp_seq=4 ttl=63 time=60.7 ms 64 bytes from 192.168.18.71: icmp_seq=5 ttl=63 time=82.0 ms 64 bytes from 192.168.18.71: icmp_seq=6 ttl=63 time=107 ms 64 bytes from 192.168.18.71: icmp_seq=7 ttl=63 time=9.48 ms --- 192.168.18.71 ping statistics --- 7 packets transmitted, 7 received, 0% packet loss, time 6010ms rtt min/avg/max/mdev = 9.479/59.777/107.016/37.445 ms lms@Z690:~/Powerwall-Dashboard$

SomermanM commented 1 year ago

PW_EMAIL=marnin@lmsconsulting.com PW_PASSWORD= PW_HOST=192.168.18.71 PW_TIMEZONE=America/Los_Angeles TZ=America/Los_Angeles PW_DEBUG=no ./pypowerwall.env (END)

jasonacox commented 1 year ago

Thanks @SomermanM - that's helpful! Fair warning, I have very limited experience with Windows. I have a workstation set up for testing but I'm no expert by any means. 😉

2023-02-11 10:46:21 pypowerwall.ConnectionError: Unable to connect to Powerwall at https://192.168.18.71/ 2023-02-11 06:03:40 urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='192.168.18.71', port=443): Read timed out. (read timeout=10)

There is something in your Win11 WSL2 setup that is blocking the connection. I see you tested the ping (ICMP) traffic, but I do wonder if it is blocking TCP 443 calls.

Can you try this from the shell that you used for ping?

# install pypowerwall module if not already installed
python3 -m pip install pypowerwall

# scan for powerwall
python3 -m pypowerwall scan

# manually connect
python3
import pypowerwall

pw = pypowerwall.Powerwall("192.168.18.71","yourpassword")

print(pw.site_name())
print(pw.status())
print(pw.home())
print(pw.grid())
SomermanM commented 1 year ago

/usr/bin/python3: No module named pip

/usr/bin/python3: No module named pypowerwall

SomermanM commented 1 year ago

But Docker is showing it running. image

jasonacox commented 1 year ago

I did a web search to see how to install python package manager (pip) on a Windows WSL system. Here is what I found:

sudo apt install python3 python3-pip
pip3 install pypowerwall
python3 -m pypowerwall scan

I'm asking to try that to see if we can narrow down the issue. This is so puzzling. Do you have any other Windows Firewall or any Security software running on your PC or network that could be blocking the TCP port access from the docker container? The fact that you can hit it from the browser on that PC makes me think that you don't.

SomermanM commented 1 year ago

Enter Network or press enter to use 172.23.142.0/24: 192.168.18.0/24

Running Scan...
  Host: 192.168.18.1 ... OPEN - Not a Powerwall
  Host: 192.168.18.35 ... OPEN - Not a Powerwall
  Host: 192.168.18.71 ... OPEN - Found Powerwall 1232100-10-H--CN322082G1H04E
                                 [Firmware 22.26.4 fc00d5dd]
  Host: 192.168.18.89 ... OPEN - Not a Powerwall
jasonacox commented 1 year ago

First of all, that's great news! Do you mind making the actual connection to ensure we aren't missing something?

# manually connect
python3

import pypowerwall

pw = pypowerwall.Powerwall("192.168.18.71","yourpassword")

print(pw.site_name())
print(pw.status())
print(pw.home())
print(pw.grid())

I suspect that works and that would mean WSL is allowing python to connect to the Powerwall. The problem seems to be something in docker that is blocking. One option to try - delete and rebuild pypowerwall container:

# stop
docker stop pypowerwall

# remove
docker rm pypowerwall
docker images | grep pypowerwall | awk '{print $3}' | xargs docker rmi -f

# rebuild
./compose-dash.sh up -d

# check logs
docker logs pypowerwall
BJReplay commented 1 year ago

Hi, also running WSL on Windows 11 (but without any issues).

Playing along just to see if I can help for some differential diagnosis. I suspect @jasonacox that your suggestion to rebuild the docker pypowerwall container will be the solution is likely to be the next best troubleshooting step.

On my WSL host:

pip3 install pypowerwall
Defaulting to user installation because normal site-packages is not writeable
Collecting pypowerwall
  Downloading pypowerwall-0.6.0-py2.py3-none-any.whl (30 kB)
Requirement already satisfied: protobuf in /usr/lib/python3/dist-packages (from pypowerwall) (3.12.4)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from pypowerwall) (2.25.1)
Installing collected packages: pypowerwall
Successfully installed pypowerwall-0.6.0

Followed by:

python3 -m pypowerwall scan

pyPowerwall Network Scanner [0.6.0]
Scan local network for Tesla Powerwall Gateways

    Your network appears to be: 172.25.98.0/24

    Enter Network or press enter to use 172.25.98.0/24:

    Running Scan...
      Host: 172.25.98.100 ... OPEN - Not a Powerwall
      Done

Discovered 0 Powerwall Gateway

However, if I ensure that it doesn't use the default WSL network...

 python3 -m pypowerwall scan

pyPowerwall Network Scanner [0.6.0]
Scan local network for Tesla Powerwall Gateways

    Your network appears to be: 172.25.98.0/24

    Enter Network or press enter to use 172.25.98.0/24: 192.168.1.0/24

    Running Scan...
      Host: 192.168.1.1 ... OPEN - Not a Powerwall
      Host: 192.168.1.30 ... OPEN - Found Powerwall 1118431-01-F--T17Hnnnnnnn
                                     [Firmware 22.26.2 8cd8cac4]
      Host: 192.168.1.101 ... OPEN - Not a Powerwall
      Done

Discovered 1 Powerwall Gateway
     192.168.1.30 [1118431-01-F--T17Hnnnnnnn] Firmware 22.26.2 8cd8cac4
python3
Python 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pypowerwall
>>> pw = pypowerwall.Powerwall("192.168.1.30","mypassword")
>>> print(pw.site_name())
BLAH's Powerwall
>>> print(pw.status())
{'din': '1118431-01-F--T17Hnnnnnnn', 'start_time': '2022-10-25 16:28:31 +0800', 'up_time_seconds': '314h31m10.712652545s', 'is_new': False, 'version': '22.26.2 8cd8cac4', 'git_hash': '8cd8cac4b9cb48bc2f25bb104eeb5d626ebbfcc2', 'commission_count': 0, 'device_type': 'hec', 'teg_type': 'unknown', 'sync_type': 'v1', 'leader': '', 'followers': None, 'cellular_disabled': False}
>>> print(pw.home())
432.7293724276218
>>> print(pw.grid())
-25.790000915527344
>>>
SomermanM commented 1 year ago

I rebuilt it from jasonacox's suggestion: Running Docker Compose... Pulling pypowerwall (jasonacox/pypowerwall:latest)... latest: Pulling from jasonacox/pypowerwall 8921db27df28: Pull complete 3fd9832a787c: Pull complete d7441962c99a: Pull complete d9d6bdefaef2: Pull complete f7ccf21cbe20: Pull complete d9cb98a4da29: Pull complete 4c3ae4e1b238: Pull complete 6b232587a89b: Pull complete Digest: sha256:876140b4325cae152f6c0c3ac555671a85a2c5d02ff1748369121db4ae74de92 Status: Downloaded newer image for jasonacox/pypowerwall:latest influxdb is up-to-date weather411 is up-to-date grafana is up-to-date Creating pypowerwall ... done telegraf is up-to-date lms@Z690:~/Powerwall-Dashboard$ docker logs pypowerwall 02/11/2023 05:14:43 PM [proxy] [INFO] pyPowerwall [0.6.0] Proxy Server [t24] - HTTP Port 8675 02/11/2023 05:14:43 PM [proxy] [INFO] pyPowerwall Proxy Started

This is what I get in the logs: During handling of the above exception, another exception occurred: 2023-02-11 17:22:01 2023-02-11 17:22:01 Traceback (most recent call last): 2023-02-11 17:22:01 File "/app/server.py", line 107, in 2023-02-11 17:22:01 pw = pypowerwall.Powerwall(host,password,email,timezone,cache_expire,timeout,pool_maxsize) 2023-02-11 17:22:01 File "/usr/local/lib/python3.10/site-packages/pypowerwall/init.py", line 138, in init 2023-02-11 17:22:01 self._get_session() 2023-02-11 17:22:01 File "/usr/local/lib/python3.10/site-packages/pypowerwall/init.py", line 151, in _get_session 2023-02-11 17:22:01 raise ConnectionError(err) 2023-02-11 17:22:01 pypowerwall.ConnectionError: Unable to connect to Powerwall at https://192.168.18.71 2023-02-11 17:22:02 02/11/2023 05:22:02 PM [proxy] [INFO] pyPowerwall [0.6.0] Proxy Server [t24] - HTTP Port 8675 2023-02-11 17:22:02 02/11/2023 05:22:02 PM [proxy] [INFO] pyPowerwall Proxy Started

I run a scan and get:

Enter Network or press enter to use 172.23.142.0/24: 192.168.18.0/24

Running Scan...
  Host: 192.168.18.1 ... OPEN - Not a Powerwall
  Host: 192.168.18.35 ... OPEN - Not a Powerwall
  Host: 192.168.18.71 ... OPEN - Found Powerwall 1232100-10-H--CN322082G1H04E
                                 [Firmware 22.26.4 fc00d5dd]
  Host: 192.168.18.89 ... OPEN - Not a Powerwall
  Done

Discovered 1 Powerwall Gateway 192.168.18.71 [1232100-10-H--CN322082G1H04E] Firmware 22.26.4 fc00d5dd

jasonacox commented 1 year ago

@BJReplay thanks for the help - this one has me stumped.

@SomermanM - thanks for trying that.

Can you try to load this page: http://192.168.18.71:8675/stats - I want to see if pypowerwall is at least listening.

Second, the scan tells us that pypowerwall is getting connectivity. But it doesn't try to authenticate. Can you still try this? I'm sorry I keep asking, but this will tell us if there is something wrong with the pypowerwall credentials (of course change yourpassword to your real Powerwall password):

$python

import pypowerwall

pw = pypowerwall.Powerwall("192.168.18.71","yourpassword")

print(pw.site_name())
print(pw.status())
print(pw.home())
print(pw.grid())

If that DOES work, then I have two other thoughts:

1) Reboot - I know that is cliche but the docker desktop stack may be in a bad state. A clean reboot may help.

2) Run pypowerwall proxy in DEBUG mode - Edit the pypowerwall.env file and change the line: PW_DEBUG=yes and restart the container with docker restart pypowerwall and debug docker logs pypowerwall -f - This may provide us with additional clues as to why it is not connecting.

BuongiornoTexas commented 1 year ago

Probably not a lot of help, but this has the feel of a firewall problem with WSL2 and the host machine. Allowing pings rhrough and grafana on port 80, but blocking 443 and any other port?

SomermanM commented 1 year ago

I gave up and installed Ubuntu in a VirtualBox I just created. The Windows WSL2 I have never used and wanted to give it a try...being uninstalled. It's up and working on the VM!! Thank you @jasonacox and @BJReplay for your help.

BJReplay commented 1 year ago

Something I did to make my life easier (years ago, when I was automating control - back in the day when you could actually locally control the Powerwall) was to install the certificate on the windows machine and set up a hosts entry so that it would work.

The certificate has a subject alternative name of powerwall, so what I did on the host where I was running a service was set up a hosts entry (and, on my router, a matching DHCP reservation).

The hosts entry is redundant if your router manages the IP address and will hand out a host name and do name resolution for you.

Since @SomermanM you're running windows, you might be able to do the same. I'm not saying this will solve your problem, but given I'm running WSL on windows, it might help :)

This, in theory, shouldn't impact on pypowerwall, because it should be working ignoring the untrusted certificate, but I only suggest it just in case there is something funky going on with docker on windows that means that pypowerwall can't ignore untrusted certificates or https connections, and I didn't experience because I happen to already have done this, which means I didn't see this issue, and that's the issue causing your time out. In any case, it means you won't see certificate warnings when you browse to your powerwall from that PC again :)

jasonacox commented 1 year ago

I'm glad you got it to work @SomermanM !!!

@BJReplay thanks for those details. Did you find that was required to run the dashboard in WSL for you?

Maybe we should add "Ubuntu VM on VirtualBox" in the instructions as a valid alternative for someone running Windows? As for WSL, @SomermanM you aren't alone. It took me 4 tries to get WSL working correctly on my Windows 11 machine. Lots of rebooting , installing, re-imaging and searching through Microsoft help... I confess, Linux and MacOS systems have spoiled me. Thankfully, with the help of the brilliant people here in this community, I finally got it to work and the dashboard setup worked perfectly. The steps I took are here: https://github.com/jasonacox/Powerwall-Dashboard#windows-11-instructions

If you ever decide to try WSL again and get it to work, let us know. In the meantime, congrats! Enjoy the dashboard!

BJReplay commented 1 year ago

Did you find that was required to run the dashboard in WSL for you?

No, I didn't (but then again, it had been done long ago to make life easier, and I had come across a regression in something else where an ignore invalid certificate option had stopped working, which is why I had had to do that, in the first place).

My only real difficulty with WSL was the annoying fact that you have to install Docker Desktop - you can't just instal the Linux components. and the docker desktop upgrade process (if initiated from the upgrade prompt within docker desktop) appears to be broken. But it's running now, and nicely enough, and so that's what I'll stick with. It is giving me the best of both worlds, running my existing windows service, and docker, and a pretty decent Linux environment.