Grafana issue on Synology NAS

jaydkay commented 2 years ago

Hi,

I tried setting up the Powerwall-Dashboard with Docker on a Synology NAS. I'm not sure if this is even supposed to work, but the setup script is running fine and three containers seem to start up okay. Only grafana keeps shutting down with the following entries in the docker log:

GF_PATHS_DATA='/var/lib/grafana' is not writable. You may have issues with file permissions, more information here: http://docs.grafana.org/installation/docker/#migration-from-a-previous-version-of-the-docker-container-to-5-1-or-later mkdir: cannot create directory '/var/lib/grafana/plugins': Permission denied

Any idea what is causing this or how to get grafana running?

Best regards, Jochen

panderson87 commented 1 year ago

Hey all, I'm installing the powerwall dashboard on my Synology NAS. Reading through this has helped me over the hurdles you all have encountered. I ran the quick start version (option 1) and was able to get all containers running. I am running into an issue that I have not seen mentioned. I am stuck on the Grafana setup portion. When adding the InfluxDB as a data source, I receive "error connecting influxDB influxQL" after I click Save and Test. I confirm the influxdb container is running along with everything else. Am i doing something wrong? influxdb log.txt

jasonacox commented 1 year ago

Hi @panderson87 - thanks for opening this.

Did you happen to record or remember the main steps you took? It would be good to have that for others following the same path. 🙏

But more important, let's see if we can get your setup working!

InfluxDB Issues

This error could mean InfluxDB is not really listening - This would be evident in the "telegraf" logs:

# check logs of telegraf
docker logs telegraf

Errors would look like:

2023-02-20T05:37:00Z W! [outputs.influxdb] When writing to [http://influxdb:8086]: database "powerwall" creation failed: Post "http://influxdb:8086/query": dial tcp 172.23.0.3:8086: connect: connection refused
2023-02-20T05:37:10Z E! [outputs.influxdb] When writing to [http://influxdb:8086]: failed doing req: Post "http://influxdb:8086/write?db=powerwall&rp=raw": dial tcp 172.23.0.3:8086: connect: connection refused
2023-02-20T05:37:10Z E! [agent] Error writing to outputs.influxdb: could not write any address
2023-02-20T05:37:20Z E! [outputs.influxdb] When writing to [http://influxdb:8086]: failed doing req: Post "http://influxdb:8086/write?db=powerwall&rp=raw": dial tcp 172.23.0.3:8086: connect: connection refused
2023-02-20T05:37:20Z E! [agent] Error writing to outputs.influxdb: could not write any address

Docker Networking Issues

The error you see could mean Grafana is unable to resolve the name "influxdb" which docker compose should handle. Option 1 (setup.sh) uses docker-compose to get this running (including setting up the 'network powerwall-dashboard_default' to handle the lookup and communication). To test that all of this is set up correctly try this:

# take the stack down
./compose-dash.sh down

# rebuild the stack
./compose-dash.sh up -d

It should show something like this:

Try to set up the database again in Grafana after restarting.

panderson87 commented 1 year ago

Thanks for the quick reply. So in checking the telegraf container, it was not fully started. Logs showed "error loading config file /etc/telegraf/telegraf.d/local.conf: open /etc/telegraf/telegraf.d/local.conf: permission denied"

So, I deleted the container, updated the powerwall.yml file and under telegraf, changed the user from 1000:1000 to match the uid/group of my Synology installer account (I initially did this for all of the other "user" entries in the file except for this one.) Reran setup and was able to get the past the issue with the influxdb powerwall database.

10 minutes in, I am monitoring the Powerwall Power Flow dashboard and only see Weather reporting in so far. All other graphs are currently showing "No Data." I will give it more time.

I'll also type up my steps i took to get this installed later this evening.

Thanks!

jasonacox commented 1 year ago

Thanks @panderson87 !

error loading config file /etc/telegraf/telegraf.d/local.conf: open /etc/telegraf/telegraf.d/local.conf: permission denied

For anyone else who has that problem, it is an easy to fix with:

chmod 644 telegraf*
docker restart telegraf

All other graphs are currently showing "No Data." I will give it more time.

You should see data from telegraf immediately so this is not a good sign. Telegraf is likely still having an error. Check docker logs telegraf and if you see the permission denied issue, try the above steps.

If you look at the directory (ls -l telegraf*) it should look similar to this (notice -rw-r--r-- permissions):

panderson87 commented 1 year ago

Looks like plugin errors.

Ran the commands above just to double-checked permissions, and they look ok. Restarted container but not working still

jasonacox commented 1 year ago

The errors indicate that pypowerwall is not running correctly (see what docker logs pypowerwall show).

Also, did you start the containers using ./compose-dash.sh up -d? I ask because docker-compose needs to build the networking to allow the containers to talk to each other. Restart would be:

./compose-dash.sh down
./compose-dash.sh up -d

panderson87 commented 1 year ago

Sorry, I restarted the containers all manually from the Docker Gui in Synology. Just rerun the commands in ssh and issue persists.

In checking the pypowerwall logs it looks like a login issue and/or network issue. it's currently on wifi and it tends to drop off the network even though the AP is literally on the other side of the wall, i should get it hardwired.

Receiving this message when trying to log into the powerwall as a customer. Error: API Limit Reached (Api Limit reached for this endpoint).

Here were the logs but I'm sure the API limit is the issue. pypowerwall.txt

marcbaier commented 1 year ago

are you sure you don't have an authorization issue ? you should be seeing data immediately once grafana is running. did you use the last 5 digits from your tesla gateway and not the whole password ?

dsf01 commented 1 year ago

I had a similar issue today where pypowerwall wouldn’t start after update. PW had an IP address but I couldn’t reach it from the browser. I ended up restarting the eero node it was connected to and then it worked.The timing was odd because it was right after I upgraded 2.8.2 > 2.8.3. On Feb 20, 2023, at 11:46 AM, panderson87 @.***> wrote: Sorry, I restarted the containers all manually from the Docker Gui in Synology. Just rerun the commands in ssh and issue persists. In checking the pypowerwall logs it looks like a login issue and/or network issue. it's currently on wifi and it tends to drop off the network even though the AP is literally on the other side of the wall, i should get it hardwired. Receiving this message when trying to log into the powerwall as a customer. Error: API Limit Reached (Api Limit reached for this endpoint). Here were the logs but I'm sure the API limit is the issue. pypowerwall.txt

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

panderson87 commented 1 year ago

are you sure you don't have an authorization issue ? you should be seeing data immediately once grafana is running. did you use the last 5 digits from your tesla gateway and not the whole password ?

Yes, I am positive but will need to double-check that I don't have a typo. I changed the default password and have been using HomeAssistant to tap into the solar/powerwall's production data Energy.

@dsf01 Yes, same issue here. Either the page does not respond, which a handful of mesh reboots will fix that, or now in this case, I'm getting the API limit error. I wonder if the HomeAssistant integration is having something to do with it.

panderson87 commented 1 year ago

Issue resolved. It did end up being the wrong password, somehow it got reset back to default! Thanks all.

Like I said I'll try to type out series of events later today.

Get Outlook for Androidhttps://aka.ms/AAb9ysg

From: dsf01 @.> Sent: Monday, February 20, 2023 3:12:35 PM To: jasonacox/Powerwall-Dashboard @.> Cc: panderson87 @.>; Mention @.> Subject: Re: [jasonacox/Powerwall-Dashboard] Grafana issue on Synology NAS (Issue #22)

I had a similar issue today where pypowerwall wouldn’t start after update. PW had an IP address but I couldn’t reach it from the browser. I ended up restarting the eero node it was connected to and then it worked.The timing was odd because it was right after I upgraded 2.8.2 > 2.8.3. On Feb 20, 2023, at 11:46 AM, panderson87 @.***> wrote: Sorry, I restarted the containers all manually from the Docker Gui in Synology. Just rerun the commands in ssh and issue persists. In checking the pypowerwall logs it looks like a login issue and/or network issue. it's currently on wifi and it tends to drop off the network even though the AP is literally on the other side of the wall, i should get it hardwired. Receiving this message when trying to log into the powerwall as a customer. Error: API Limit Reached (Api Limit reached for this endpoint). Here were the logs but I'm sure the API limit is the issue. pypowerwall.txt

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

— Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjasonacox%2FPowerwall-Dashboard%2Fissues%2F22%23issuecomment-1437503941&data=05%7C01%7C%7C9853cda7690a42bbd55408db137ecf0c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638125207590930188%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=bRK%2BYz7wB8ByTJDRRbbDx6sEgTPbcG%2BL0GY0EkvYUcg%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAXCOJGDKIZNIRB5PYACS4Y3WYPF3HANCNFSM5UACDZBQ&data=05%7C01%7C%7C9853cda7690a42bbd55408db137ecf0c%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638125207590930188%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2SThAEfrb7Okn3vMS2iTLVi%2BbNL4foHjxcAH%2Fs3wBOw%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>

BuongiornoTexas commented 1 year ago

@panderson87 - Your powerwall dropping off wifi may be an ARP problem - more specifically, the powerwall may still be on the network, but ignoring ARP requests.

For me the symptoms were:

I could access the powerwall web interface immediately after rebooting my router.
After a couple of hours, it disappeared from the network, and I had to use the app to "reconnect" to the wifi.
Powerwall-Dashboard worked fine until the server machine needed a reboot, after which it lost the powerwall data feed.

If your problem is similar, you could try adding a static ARP entry - worked like magic for me (and saved several hundred dollars for the hard wiring).

panderson87 commented 1 year ago

I'll give that a shot. I had a dhcp reservation set up in the router and even forced the pw to connect to 2.4ghz thinking that would've help.

Get Outlook for Androidhttps://aka.ms/AAb9ysg

From: BuongiornoTexas @.> Sent: Monday, February 20, 2023 4:20:19 PM To: jasonacox/Powerwall-Dashboard @.> Cc: panderson87 @.>; Mention @.> Subject: Re: [jasonacox/Powerwall-Dashboard] Grafana issue on Synology NAS (Issue #22)

@panderson87https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fpanderson87&data=05%7C01%7C%7C3cf08625780e43b9e66008db13884530%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638125248218179281%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YBKIkAMBYFwX%2FvQiUd90NbCZS31lH8ORfyvLC2w77IU%3D&reserved=0 - Your powerwall dropping off wifi may be an ARP problem - more specifically, the powerwall may still be on the network, but ignoring ARP requests.

For me the symptoms were:

I could access the powerwall web interface immediately after rebooting my router.
After a couple of hours, it disappeared from the network, and I had to use the app to "reconnect" to the wifi.
Powerwall-Dashboard worked fine until the server machine needed a reboot, after which it lost the powerwall data feed.

If your problem is similar, you could try adding a static ARP entry - worked like magic for me (and saved several hundred dollars for the hard wiring).

— Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fjasonacox%2FPowerwall-Dashboard%2Fissues%2F22%23issuecomment-1437559105&data=05%7C01%7C%7C3cf08625780e43b9e66008db13884530%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638125248218179281%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=foBQ3S7JxPmY2eSki0CE02ijUQdbWsqYavrP4bM1Erw%3D&reserved=0, or unsubscribehttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAXCOJGDMT5GXPUTDMYSDXWDWYPNZHANCNFSM5UACDZBQ&data=05%7C01%7C%7C3cf08625780e43b9e66008db13884530%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638125248218179281%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=CGsNhDFwSGg1Ous%2FzaOYgKf6lRo1xgpKosUOgDz6qA8%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.***>

BuongiornoTexas commented 1 year ago

Does sound like my issue (I also tried the 2.4GHz trick without success) - if the static ARP works, I've got a couple of cheat sheets for permanent static ARPs for windows and systemd setups. Let me know if you want either of them.

jasonacox commented 1 year ago

Congrats on getting it to work @panderson87 !

It did end up being the wrong password, somehow it got reset back to default!

I've seen that before. I don't know what triggers it, but have seen firmware updates reset the password in the past. The "Error: API Limit Reached" is the result of multiple bad password attempts.

On the WiFi conversation, I gave up on that after always getting drop-outs (even though it would reconnect after 5-10m) and just ran a hardware. Zero issues for past year because of that. But any WiFI pro tips @BuongiornoTexas would be great for our community. :)

I had a thought... while the issues seem to be more prevalent for Synology users, we seem to have an increasing number of users having setup problems. Most are related to some odd file ownership or permission setting (or even auth issue like this one for @panderson87 ). We spend a lot of time going back and forth doing basic troubleshooting steps to see what part of the stack isn't working correctly. It would be super amazing if we had a simple verify.sh (or troubleshoot.sh) script that basically tests all of the components for health and reports exactly which parts of the stack are working correctly or where it sees errors. I may take a stab at it today if I have time. I feel like it would help users find problems and get their setup working faster.

BuongiornoTexas commented 1 year ago

But any WiFI pro tips @BuongiornoTexas would be great for our community. :)

Well, I did the ARP fixes around December 2022 and haven't had any issues since, so it definitely worked for my problem. @jasonacox, your issues sounds a little different - the failure to respond to ARP takes a bit longer than 5-10 minutes to appear.

I'm going to save myself a bit of time - here is the write up I did for whirlpool. https://whrl.pl/RgxRzh

panderson87 commented 1 year ago

Here are the steps that I took to get this working on my NAS. I'm a github and noob so sorry if the formatting is not correct.

Make sure Docker has been installed in Package Center and SSH is enabled in Terminal & SNMP on your Synology NAS

On Synology NAS, in Control Panel>Shared Folder> Create a shared folder named powerwall
On Synology NAS, in Control Panel>User & Groups, select your user and edit. Under Permissions, make sure your user has read/write permissions to the powerwall share you created.
On your computer, from a browser, navigate to https://github.com/jasonacox/Powerwall-Dashboard.git
Download zip file and extract source code.
From your computer, in Windows Explorer, navigate to \YourNasIP\powerwall then copy and paste the extracted source code into the powerwall share. Confirm both files AND folders are there.
Use your favorite SSH tool and log in to your NAS (I used Putty) and log in as your install user (not root.) type in “id” and press enter to find your uid and gid. Back on your NAS, edit the powerwall.yml file located in the powerwall share. Find all lines that say: user: "1000:1000" and replace with your uid and gid and save. In my case, mine now says user: "1026:101" I changed this for pypowerwall, telegraf, Grafana, and weather411.
Back in SSH navigate to the Powerwall share where the source code is extracted to and run the command ./setup.sh and follow the steps in the interactive setup. If the setup does not run due to docker errors, refer to https://github.com/jasonacox/Powerwall-Dashboard#docker-errors

jasonacox commented 1 year ago

Wow! Thanks @panderson87 !

jaydkay commented 1 year ago

In my case, in addition to the steps regarding file ownership of docker and the usergroup settings mentioned in this post, I've only changed the user setting for grafana in compose.env. I did not need to change the user settings for all other containers, i.e. pypowerwall, telegraf and weather411. Don't know if it is important, just thought I should mention it.

jasonacox commented 1 year ago

Thanks @jaydkay - I'll keep this open to remind me to combine this insight into a guidance doc and FAQ for Synology users (unless someone else is willing to do that and submit a PR 😉 ).

I did not need to change the user settings for all other containers, i.e. pypowerwall, telegraf and weather411

Correct, they only read from the file system so as long as their user has read permission, it will work. Grafana needs to write to disk for its settings.

danieltian commented 1 year ago

I got this working following this comment by @jaydkay, but I wanted to clarify how to update the user in case it'll help someone else. After ssh'ing into the Synology NAS and running id, you will see this:

uid=1026(Daniel) gid=100(users) groups=100(users),101(administrators),65538(docker)

Open powerwall.yml and change every place where user: appears to the uid and group of your user, e.g. user: "1026:1026". There should be 4 places: under pypowerwall, telegraf, grafana, and weather411. The grafana entry will be user: "${GRAFANAUSER}", but just change it to user: "1026:1026". Of course, use the your own uid, not mine :).

jasonacox commented 1 year ago

Thank you, Daniel!

jasonacox / Powerwall-Dashboard

Grafana issue on Synology NAS #22

InfluxDB Issues

Docker Networking Issues