search

jasonacox / Powerwall-Dashboard

Grafana Monitoring Dashboard for Tesla Solar and Powerwall Systems
MIT License
292 stars 59 forks source link

Repo cleanup.. remove ignored files, update ignore list #265

Open YesThatAllen opened 1 year ago

YesThatAllen commented 1 year ago

this is an offer for me to help, I'm looking for confirmation & maybe some guidance

Problem

I'm seeing a number of changes in my local git gui which could be prevented with some cleanup.

Enhancement

Additional context

I've been running cross platform dev teams for a while. Not everyone knows about a .gitattributes file or .editorconfig. I'm not super familiar with how to help the actionable code, but I can sure clean up a repo. 😄

I plan on opening a PR with changes soon, I'll have some questions about how best stuff should work.

My local diff in the moment:

2023-05-02_10-12-57

jasonacox commented 1 year ago

Hi @YesThatAllen - I would love a PR for this! Thank for your help. 🙏

YesThatAllen commented 1 year ago

Here's a start to the work.. all feedback is welcome.

I've got a question regarding timezones.. I wonder if the diff I'm about to paste in could be handled in a separate issue regarding timezone support, but maybe I'm setting my tz to Chicago the wrong way, resulting in the diff.

I'm mostly curious about the change in the /influxdb/influxdb.sql file

TZ related diff I still have this diff after running the update script ```diff % git --no-pager diff diff --git a/dashboard.json b/dashboard.json index b09a845..cadbd04 100644 --- a/dashboard.json +++ b/dashboard.json @@ -34,7 +34,7 @@ "name": "VAR_TZ", "type": "constant", "label": "Timezone", - "value": "America/Los_Angeles", + "value": "America/Chicago", "description": "" } ], diff --git a/dashboards/dashboard-min-mean-max.json b/dashboards/dashboard-min-mean-max.json index 5af17d6..052f288 100644 --- a/dashboards/dashboard-min-mean-max.json +++ b/dashboards/dashboard-min-mean-max.json @@ -27,7 +27,7 @@ "name": "VAR_TZ", "type": "constant", "label": "Timezone", - "value": "America/Los_Angeles", + "value": "America/Chicago", "description": "" } ], diff --git a/dashboards/dashboard-no-animation.json b/dashboards/dashboard-no-animation.json index 4bac66e..da5b73c 100644 --- a/dashboards/dashboard-no-animation.json +++ b/dashboards/dashboard-no-animation.json @@ -27,7 +27,7 @@ "name": "VAR_TZ", "type": "constant", "label": "Timezone", - "value": "America/Los_Angeles", + "value": "America/Chicago", "description": "" } ], diff --git a/dashboards/dashboard-simple.json b/dashboards/dashboard-simple.json index 6fcd549..d61d870 100644 --- a/dashboards/dashboard-simple.json +++ b/dashboards/dashboard-simple.json @@ -27,7 +27,7 @@ "name": "VAR_TZ", "type": "constant", "label": "Timezone", - "value": "America/Los_Angeles", + "value": "America/Chicago", "description": "" } ], diff --git a/dashboards/dashboard.json b/dashboards/dashboard.json index b09a845..cadbd04 100644 --- a/dashboards/dashboard.json +++ b/dashboards/dashboard.json @@ -34,7 +34,7 @@ "name": "VAR_TZ", "type": "constant", "label": "Timezone", - "value": "America/Los_Angeles", + "value": "America/Chicago", "description": "" } ], diff --git a/influxdb/influxdb.sql b/influxdb/influxdb.sql index 3beb4b4..99976e7 100644 --- a/influxdb/influxdb.sql +++ b/influxdb/influxdb.sql @@ -13,8 +13,8 @@ CREATE RETENTION POLICY grid ON powerwall duration INF replication 1 CREATE RETENTION POLICY pod ON powerwall duration INF replication 1 CREATE RETENTION POLICY alerts ON powerwall duration INF replication 1 CREATE CONTINUOUS QUERY cq_autogen ON powerwall BEGIN SELECT mean(home) AS home, mean(solar) AS solar, mean(from_pw) AS from_pw, mean(to_pw) AS to_pw, mean(from_grid) AS from_grid, mean(to_grid) AS to_grid, last(percentage) AS percentage INTO powerwall.autogen.:MEASUREMENT FROM (SELECT load_instant_power AS home, solar_instant_power AS solar, abs((1+battery_instant_power/abs(battery_instant_power))*battery_instant_power/2) AS from_pw, abs((1-battery_instant_power/abs(battery_instant_power))*battery_instant_power/2) AS to_pw, abs((1+site_instant_power/abs(site_instant_power))*site_instant_power/2) AS from_grid, abs((1-site_instant_power/abs(site_instant_power))*site_instant_power/2) AS to_grid, percentage FROM raw.http) GROUP BY time(1m), month, year fill(linear) END -CREATE CONTINUOUS QUERY cq_kwh ON powerwall RESAMPLE EVERY 1m BEGIN SELECT integral(home)/1000/3600 AS home, integral(solar)/1000/3600 AS solar, integral(from_pw)/1000/3600 AS from_pw, integral(to_pw)/1000/3600 AS to_pw, integral(from_grid)/1000/3600 AS from_grid, integral(to_grid)/1000/3600 AS to_grid INTO powerwall.kwh.:MEASUREMENT FROM autogen.http GROUP BY time(1h), month, year tz('America/Los_Angeles') END -CREATE CONTINUOUS QUERY cq_daily ON powerwall RESAMPLE EVERY 1h BEGIN SELECT sum(home) AS home, sum(solar) AS solar, sum(from_pw) AS from_pw, sum(to_pw) AS to_pw, sum(from_grid) AS from_grid, sum(to_grid) AS to_grid INTO powerwall.daily.:MEASUREMENT FROM powerwall.kwh.http GROUP BY time(1d), month, year tz('America/Los_Angeles') END +CREATE CONTINUOUS QUERY cq_kwh ON powerwall RESAMPLE EVERY 1m BEGIN SELECT integral(home)/1000/3600 AS home, integral(solar)/1000/3600 AS solar, integral(from_pw)/1000/3600 AS from_pw, integral(to_pw)/1000/3600 AS to_pw, integral(from_grid)/1000/3600 AS from_grid, integral(to_grid)/1000/3600 AS to_grid INTO powerwall.kwh.:MEASUREMENT FROM autogen.http GROUP BY time(1h), month, year tz('America/Chicago') END +CREATE CONTINUOUS QUERY cq_daily ON powerwall RESAMPLE EVERY 1h BEGIN SELECT sum(home) AS home, sum(solar) AS solar, sum(from_pw) AS from_pw, sum(to_pw) AS to_pw, sum(from_grid) AS from_grid, sum(to_grid) AS to_grid INTO powerwall.daily.:MEASUREMENT FROM powerwall.kwh.http GROUP BY time(1d), month, year tz('America/Chicago') END CREATE CONTINUOUS QUERY cq_monthly ON powerwall RESAMPLE EVERY 1h BEGIN SELECT sum(home) AS home, sum(solar) AS solar, sum(from_pw) AS from_pw, sum(to_pw) AS to_pw, sum(from_grid) AS from_grid, sum(to_grid) AS to_grid INTO powerwall.monthly.:MEASUREMENT FROM powerwall.daily.http GROUP BY time(365d), month, year END CREATE CONTINUOUS QUERY cq_pw_temps ON powerwall BEGIN SELECT mean(PW1_temp) AS PW1_temp, mean(PW2_temp) AS PW2_temp, mean(PW3_temp) AS PW3_temp, mean(PW4_temp) AS PW4_temp, mean(PW5_temp) AS PW5_temp, mean(PW6_temp) AS PW6_temp INTO powerwall.pwtemps.:MEASUREMENT FROM (SELECT PW1_temp, PW2_temp, PW3_temp, PW4_temp, PW5_temp, PW6_temp FROM raw.http) GROUP BY time(1m), month, year fill(linear) END CREATE CONTINUOUS QUERY cq_pw_tempsb ON powerwall BEGIN SELECT mean(PW7_temp) AS PW7_temp, mean(PW8_temp) AS PW8_temp, mean(PW9_temp) AS PW9_temp, mean(PW10_temp) AS PW10_temp, mean(PW11_temp) AS PW11_temp, mean(PW12_temp) AS PW12_temp INTO powerwall.pwtemps.:MEASUREMENT FROM (SELECT PW7_temp, PW8_temp, PW9_temp, PW10_temp, PW11_temp, PW12_temp FROM raw.http) GROUP BY time(1m), month, year fill(linear) END diff --git a/telegraf.conf b/telegraf.conf index b7f786f..a702cab 100644 --- a/telegraf.conf +++ b/telegraf.conf @@ -115,12 +115,12 @@ [[processors.date]] tag_key = "month" date_format = "Jan" - timezone = "America/Los_Angeles" + timezone = "America/Chicago" [[processors.date]] tag_key = "year" date_format = "2006" - timezone = "America/Los_Angeles" + timezone = "America/Chicago" # Alert Data [[inputs.http]] diff --git a/tz b/tz index 49244fc..c7aac92 100644 --- a/tz +++ b/tz @@ -1 +1 @@ -America/Los_Angeles +America/Chicago % ```
jasonacox commented 1 year ago

The tz.sh script updates all those files to have your timezone and is run during setup.sh. The updated.sh uses tz.sh to reset to default (yes, America/Los_Angeles) and then does a git pull and runs tz.sh again to set it back to your local timezone as stored in the local (untracked) tz file.

Yes, it is a hack, but the timezone localization required in all those files didn't allow for an external reference.

YesThatAllen commented 1 year ago

Gotcha.. Once we're done with #269, there will be FAR fewer diffs showing, and in my mind, those will be easier to mentally ignore.

Addressing the "hack" is fodder for a later date/issue.

jaydkay commented 1 year ago

Unfortunately, I can't update to the current version due to the following reason:

admin@ds620slim:~/Powerwall-Dashboard$ ./upgrade.sh 
Upgrade Powerwall-Dashboard from 2.9.3 to 2.9.6
---------------------------------------------------------------------
This script will attempt to upgrade you to the latest version without
removing existing data. A backup is still recommended.

Upgrade - Proceed? [y/N] y

Resetting Timezone to Default...

Pull influxdb.sql, dashboard.json, telegraf.conf, and other changes...

No local changes to save
remote: Enumerating objects: 79, done.
remote: Counting objects: 100% (79/79), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 79 (delta 45), reused 63 (delta 37), pack-reused 0
Unpacking objects: 100% (79/79), 23.43 KiB | 193.00 KiB/s, done.
From https://github.com/jasonacox/Powerwall-Dashboard
   864bcde..da239f4  main       -> origin/main
 * [new tag]         v2.9.6     -> v2.9.6
 * [new tag]         v2.9.4     -> v2.9.4
 * [new tag]         v2.9.5     -> v2.9.5
Updating 864bcde..da239f4
error: unable to create file influxdb/backups/.keep: Permission denied

Any idea what might be causing this (the last upgrades never failed)?

Some files in ~/Powerwall-Dashboard/influxdb belong to the user root, but as far as I understood, I'm not supposed to run ./upgrade as root:

admin@ds620slim:~/Powerwall-Dashboard/influxdb$ ll
total 60
drwxr-xr-x 1 admin users   358 May  7 14:41 .
drwxr-xr-x 1 admin users   948 May  7 14:41 ..
drwxrwxr-x 1 root  root  14382 Feb 11 15:33 backups
drwxr-xr-x 1 root  root     36 Apr 26  2022 data
-rw-r--r-- 1 admin users  1057 Feb 20 11:18 dropcq.sql
-rw-r--r-- 1 admin users 15280 May  7 14:41 influxdb.sql
-rw-r--r-- 1 admin users 15280 May  7 14:41 influxdb.sql.bak
drwxr-xr-x 1 root  root     14 May  7 02:28 meta
-rw-r--r-- 1 admin users  1089 Feb 11 16:43 run-once-2.8.0.sql
-rw-r--r-- 1 admin users     3 Feb 11 16:46 run-once-2.8.0.sql.done
-rw-r--r-- 1 admin users   508 Feb 20 11:18 run-once-2.8.3.sql
-rw-r--r-- 1 admin users     3 Feb 20 11:18 run-once-2.8.3.sql.done
-rw-r--r-- 1 admin users  3590 May  2 17:52 run-once-2.9.2.sql
-rw-r--r-- 1 admin users     3 May  2 17:52 run-once-2.9.2.sql.done
drwx------ 1 root  root     36 Apr 26  2022 wal

Is it possible that my installation is now corrupted since ./upgrade has already started doing some changes when it failed? I guess that at least the timezone has been changed?

YesThatAllen commented 1 year ago

Any idea what might be causing this (the last upgrades never failed)? Some files in ~/Powerwall-Dashboard/influxdb belong to the user root,

Yep influxdb/backups/ was created by root, so the update script can't create a file in it.

but as far as I understood, I'm not supposed to run ./upgrade as root:

Correct. there are a number of ways we might handle this, but the simplest fix for now is to not attempt to create that .keep file. I've submitted a fix in #272

YesThatAllen commented 1 year ago

Is it possible that my installation is now corrupted since ./upgrade has already started doing some changes when it failed?

The git operations were called before anything too drastic was attempted on the local setup, so your data should be fine.

(I assume your dashboard is still running and that you can see the grafana interface?)

I guess that at least the timezone has been changed?

The upgrade script should set it back to the timezone you've specified in pypowerwall.env.

jasonacox commented 1 year ago

Merged!

@jaydkay thanks for opening this! This shows a gap in test rigs where I do not have active backup scripts running that change permissions. You should be able to run upgrade.sh again.

Minor correction on timezone - It is set in the tz file when you first ran setup.sh and upgrade will set it back to that after the new release pull.

jasonacox commented 1 year ago

@jaydkay I believe I was able to replicate a setup similar to yours (with backup owned by root) and the latest upgrade worked without error.

jaydkay commented 1 year ago

Hi @YesThatAllen and @jasonacox, thanks a lot for your quick support! Unfortunately, probably due to the failed upgrade from 2.9.3 to 2.9.6 I now get the following error:

admin@ds620slim:~/Powerwall-Dashboard$ ./upgrade.sh 
Upgrade Powerwall-Dashboard from 2.9.6 to 2.9.7
---------------------------------------------------------------------
This script will attempt to upgrade you to the latest version without
removing existing data. A backup is still recommended.

Upgrade - Proceed? [y/N] y

Resetting Timezone to Default...

Pull influxdb.sql, dashboard.json, telegraf.conf, and other changes...

Saved working directory and index state WIP on main: 864bcde v2.9.3
error: cannot pull with rebase: You have unstaged changes.
error: please commit or stash them.

The upgrade seems to think that I got 2.9.6 installed, but that upgrade failed, so I should still be with 2.9.3. Any idea how to proceed from here?

YesThatAllen commented 1 year ago

try starting with git checkout main

and if git pull --rebase doesn't succeed, review what it says is blocking.

you may need to use git checkout to reset a file or two. Post back if there's issues you can't readily get past, or with other findings?

jaydkay commented 1 year ago

Okay, so:

admin@ds620slim:~/Powerwall-Dashboard$ git checkout main
M       tools/influxdb2/flux/tesla_energy.strings.flux
M       tools/influxdb2/flux/tesla_energy.vitals.flux
Already on 'main'
Your branch is behind 'origin/main' by 15 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)

and

admin@ds620slim:~/Powerwall-Dashboard$ git pull --rebase
error: cannot pull with rebase: You have unstaged changes.
error: please commit or stash them.

Sorry, I have no idea how to fix this...

YesThatAllen commented 1 year ago

try git status ?

jaydkay commented 1 year ago

Okay, this gives:

admin@ds620slim:~/Powerwall-Dashboard$ git status
On branch main
Your branch is behind 'origin/main' by 15 commits, and can be fast-forwarded.
  (use "git pull" to update your local branch)

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   tools/influxdb2/flux/tesla_energy.strings.flux
        modified:   tools/influxdb2/flux/tesla_energy.vitals.flux

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        .editorconfig
        .gitattributes
        backups/backup.sh
        compose.env
        dashboards/archive/
        grafana.env
        grafana/.keep
        grafana/alerting/
        influxdb/backups/
        influxdb/run-once-2.8.0.sql.done
        influxdb/run-once-2.8.3.sql.done
        influxdb/run-once-2.9.2.sql.done
        telegraf.local
        tmp.sh
        tools/solar-only/grafana/.keep
        tools/solar-only/influxdb/.keep
        tools/tesla-history/tesla-history.auth
        tools/tesla-history/tesla-history.conf
        v6.5.1/grafana/.keep
        v6.5.1/influxdb/.keep

no changes added to commit (use "git add" and/or "git commit -a")
YesThatAllen commented 1 year ago

oh.. this is about line endings. you've got the new .gitattributes file which makes your local think the two .flux files have been changed.

rm .gitattributes

and try pulling again?

jaydkay commented 1 year ago

Okay, removed the file and ran git pull --rebase, which now gives:

admin@ds620slim:~/Powerwall-Dashboard$ git pull --rebase
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (3/3), done.
remote: Total 9 (delta 5), reused 8 (delta 5), pack-reused 0
Unpacking objects: 100% (9/9), 1.51 KiB | 155.00 KiB/s, done.
From https://github.com/jasonacox/Powerwall-Dashboard
   da239f4..8dd0f81  main       -> origin/main
Updating 864bcde..8dd0f81
error: The following untracked working tree files would be overwritten by merge:
        .editorconfig
        dashboards/archive/dashboard-animation.json
        dashboards/archive/dashboard-grid.json
        dashboards/archive/dashboard-new.json
        dashboards/archive/dashboard-orig.json
        dashboards/archive/dashboard-v2.8.2.json
        dashboards/archive/dashboard-v2.8.3.json
        dashboards/archive/dashboard-v2.9.0.json
        dashboards/archive/dashboard-v2.9.1.json
        grafana/.keep
        tools/solar-only/grafana/.keep
        tools/solar-only/influxdb/.keep
        v6.5.1/grafana/.keep
        v6.5.1/influxdb/.keep
Please move or remove them before you merge.
Aborting
YesThatAllen commented 1 year ago

Gotcha.. so yeah you'd want to remove those as well.

For others playing the home game; the following command, run from the repo's root, would be the thing:

rm rf .gitattributes .editorconfig dashboards/archive/dashboard-animation.json dashboards/archive/dashboard-grid.json dashboards/archive/dashboard-new.json dashboards/archive/dashboard-orig.json dashboards/archive/dashboard-v2.8.2.json dashboards/archive/dashboard-v2.8.3.json dashboards/archive/dashboard-v2.9.0.json dashboards/archive/dashboard-v2.9.1.json grafana/.keep tools/solar-only/grafana/.keep tools/solar-only/influxdb/.keep v6.5.1/grafana/.keep v6.5.1/influxdb/.keep

(leave out the -rf to confirm operations & you'll just have to hit "ok" a bunch)

jaydkay commented 1 year ago

Okay, seems we're getting closer... ;) Deleted all the files and tried again:

admin@ds620slim:~/Powerwall-Dashboard$ git pull --rebase
Updating 864bcde..8dd0f81
Fast-forward
 .editorconfig                                            |  21 +++++++++++++++++++++
 .gitattributes                                           |  25 +++++++++++++++++++++++++
 .github/ISSUE_TEMPLATE/other.md                          |   2 --
 .gitignore                                               |  35 +++++++++++++++++++++++++++++++----
 README.md                                                |  17 +++++++----------
 RELEASE.md                                               |  29 ++++++++++++++++++++++++-----
 VERSION                                                  |   2 +-
 _config.yml                                              |   2 +-
 compose.env.sample                                       |   6 +++---
 dashboard.json                                           |  58 +++++++++++++++++++++++++++++++++++++++++++---------------
 dashboards/README.md                                     |   3 ---
 dashboards/{ => archive}/dashboard-animation.json        |   0
 dashboards/{ => archive}/dashboard-grid.json             |   0
 dashboards/{ => archive}/dashboard-new.json              |   0
 dashboards/{ => archive}/dashboard-orig.json             |   0
 dashboards/{ => archive}/dashboard-v2.8.2.json           |   0
 dashboards/{ => archive}/dashboard-v2.8.3.json           |   0
 dashboards/{ => archive}/dashboard-v2.9.0.json           |   0
 dashboards/{ => archive}/dashboard-v2.9.1.json           |   0
 dashboards/dashboard.json                                |  58 +++++++++++++++++++++++++++++++++++++++++++---------------
 dashboards/powerflow-animation.html                      |   6 +++---
 grafana/.DS_Store                                        | Bin 6148 -> 0 bytes
 grafana/.keep                                            |   1 +
 grafana/README.md                                        |   2 --
 influxdb.conf                                            |   1 -
 influxdb/.DS_Store                                       | Bin 6148 -> 0 bytes
 influxdb/README.md                                       |   2 --
 location.html                                            |   1 -
 powerwall.yml                                            |  11 +++++------
 pypowerwall.sample                                       |   5 -----
 telegraf.local.sample                                    |   4 ++--
 tools/ecowitt-weather-history/ecowitt-weather-history.py |   6 +++---
 tools/influxdb2/flux/tesla_energy.strings.flux           |  52 ++++++++++++++++++++++++++--------------------------
 tools/influxdb2/flux/tesla_energy.vitals.flux            |  66 +++++++++++++++++++++++++++++++++---------------------------------
 tools/solar-only/grafana/.keep                           |   1 +
 tools/solar-only/grafana/README.md                       |   2 --
 tools/solar-only/influxdb/.keep                          |   1 +
 tools/solar-only/influxdb/README.md                      |   2 --
 upgrade.sh                                               |  22 +++++++++++-----------
 v6.5.1/grafana/.DS_Store                                 | Bin 6148 -> 0 bytes
 v6.5.1/grafana/.keep                                     |   1 +
 v6.5.1/grafana/README.md                                 |   2 --
 v6.5.1/influxdb/.DS_Store                                | Bin 6148 -> 0 bytes
 v6.5.1/influxdb/.keep                                    |   1 +
 v6.5.1/influxdb/README.md                                |   2 --
 v6.5.1/pypowerwall.sample                                |   5 -----
 verify.sh                                                |   4 ++--
 47 files changed, 289 insertions(+), 169 deletions(-)
 create mode 100644 .editorconfig
 create mode 100644 .gitattributes
 rename dashboards/{ => archive}/dashboard-animation.json (100%)
 rename dashboards/{ => archive}/dashboard-grid.json (100%)
 rename dashboards/{ => archive}/dashboard-new.json (100%)
 rename dashboards/{ => archive}/dashboard-orig.json (100%)
 rename dashboards/{ => archive}/dashboard-v2.8.2.json (100%)
 rename dashboards/{ => archive}/dashboard-v2.8.3.json (100%)
 rename dashboards/{ => archive}/dashboard-v2.9.0.json (100%)
 rename dashboards/{ => archive}/dashboard-v2.9.1.json (100%)
 delete mode 100644 grafana/.DS_Store
 create mode 100644 grafana/.keep
 delete mode 100644 grafana/README.md
 delete mode 100644 influxdb/.DS_Store
 delete mode 100644 influxdb/README.md
 delete mode 100644 pypowerwall.sample
 create mode 100644 tools/solar-only/grafana/.keep
 delete mode 100644 tools/solar-only/grafana/README.md
 create mode 100644 tools/solar-only/influxdb/.keep
 delete mode 100644 tools/solar-only/influxdb/README.md
 delete mode 100644 v6.5.1/grafana/.DS_Store
 create mode 100644 v6.5.1/grafana/.keep
 delete mode 100644 v6.5.1/grafana/README.md
 delete mode 100644 v6.5.1/influxdb/.DS_Store
 create mode 100644 v6.5.1/influxdb/.keep
 delete mode 100644 v6.5.1/influxdb/README.md
 delete mode 100644 v6.5.1/pypowerwall.sample

I suppose I can now try to normally invoke the ./upgrade.sh script? Thank you!

YesThatAllen commented 1 year ago

Yep, the script had failed on the git pull --rebase. You've done that now so the upgrade script should work as expected.

jasonacox commented 1 year ago

Upgrade will show same version, but you want to continue (technically you should always be able to re-install same version). Also, can you run ./verify.sh to ensure everything is still set up and running correctly?

@YesThatAllen should we add anything to upgrade.sh to accomodate for the repo changes?

jaydkay commented 1 year ago

Yes, ./upgrade.sh made an update from 2.9.7 to 2.9.7 and completed without errors. Also, ./verify.shshows that everything is working as expected. Thanks a lot to both of you!

YesThatAllen commented 1 year ago

@YesThatAllen should we add anything to upgrade.sh to accommodate for the repo changes?

The change in 2.9.7 which removed .keep from backups/ should be enough to prevent anyone else running in to this issue.

Other changes to the upgrade & backup scripts are fodder for separate issues.

YesThatAllen commented 1 year ago

I started putting my thoughts separate issues:

I imagine we can close this issue once it goes a while without addition related issues like @jaydkay pointed out.

YesThatAllen commented 1 year ago

Another benefit I hoped to get from this cleanup was to remove the pretty incessant logging of this statement in the grafana docker container:

="ignoring unexpected file while scanning local working directory" filename=/var/lib/grafana/alerting/.DS_Store

I ended up git-pulling and updating my local without the script, then updating docker itself, so the containers restarted. I was still seeing that error.. I don't know enough about how docker populates its containers to know if the .DS_Store file should be expected to get removed.

I was able to remove it manually by using the terminal in the container & running

rm /var/lib/grafana/alerting/.DS_Store

What I actually did ![2023-05-12_12-16-38](https://github.com/jasonacox/Powerwall-Dashboard/assets/2879972/7feebccd-7263-4f0e-9679-74d266a900a4) ``` $ ls -al /var/lib/grafana/alerting/ total 8 drwxr-x--- 4 1000 1000 128 May 5 17:24 . drwxr-xr-x 8 1000 1000 256 May 12 17:04 .. -rw-r--r-- 1 1000 1000 6148 May 5 17:24 .DS_Store drwxr-x--- 3 1000 1000 96 May 12 17:03 1 $ rm /var/lib/grafana/alerting/.DS_Store $ ls -al /var/lib/grafana/alerting/ total 0 drwxr-x--- 3 1000 1000 96 May 12 17:07 . drwxr-xr-x 8 1000 1000 256 May 12 17:04 .. drwxr-x--- 3 1000 1000 96 May 12 17:03 1 $ ```

I assume that no one who's starting this project fresh would see that message. Woot?

jgleigh commented 1 year ago

I noticed we still have a dashboard.json at the top level and a slightly newer copy at dashboards/dashboard.json. Should we cleanup the one at the top level and keep them all in the dashboards folder?

jasonacox commented 1 year ago

The original purpose of the dashboards folder was to collect user submitted "Alternative Dashboards" as the README.md says. If we move the default dashboard there, we would need to update the setup instructions in setup.sh, upgrade.sh and Readme. I have been reluctant to do that in order to make the setup easy (keeping what you need in the root folder) but the duplicate file is probably a good reason to reconsider.

jgleigh commented 1 year ago

I'm fine leaving it in the root folder, but then we should remove the duplicate in the dashboards folder. The one in dashboards had some slight tweaks to two of the queries.

jasonacox commented 1 year ago

I think that proves that we should clean it up and just use the one in the dashboards folder. It is logical and shouldn't be difficult for the community to figure out. I updated the instructions including the setup.sh and upgrade.sh references to the new location. Thanks for raise this @jgleigh ! 🙏

rcasta74 commented 1 year ago

The tz.sh script updates all those files to have your timezone and is run during setup.sh. The updated.sh uses tz.sh to reset to default (yes, America/Los_Angeles) and then does a git pull and runs tz.sh again to set it back to your local timezone as stored in the local (untracked) tz file.

Yes, it is a hack, but the timezone localization required in all those files didn't allow for an external reference.

What about to use template files and create the final ones from them? The template files would have $TZ as reference, and then envsubst command could be used to change it and write the final files.