InfluxDB error when restoring from backup

[jaydkay]

Hi @jasonacox,

I have been using Powerwall-Dashboard for some time now and today decided to upgrade to the current version. My installaton has not been updated since about july 2022.

So I first made a backup of InfluxDB according to the notes in the backup folder, basically:

docker exec influxdb influxd backup -database powerwall /var/lib/influxdb/backups sudo tar -zcvf Powerwall-Dashboard.tgz influxdb

Then I cloned the current repository, set it up from scratch using ./setup.sh and checked everything was working (all containers started up fine). As far as I can tell, everything was working fine (of course, all historical data was still missing).

Next I restored the backup, following your notes again:

./compose-dash.sh stop tar -zxvf Powerwall-Dashboard.tgz ./compose-dash.sh start

Now the historical data was available again, as well as the live data in the Power Flow diagram or the Power Meters diagram.

Unfortunately, the Energy Usage graph was now almost empty (only showing the Sun and Moon data), with a small red exclamation mark in the upper left corner, saying:

InfluxDB: InfluxDB Error: not executed

Any idea what is wrong? At the moment I have it running again without the historical data, but maybe there is an easy way to have this issue fixed.

[jasonacox]

Hi @jaydkay - thanks for opening this issue.

Did you try the upgrade.sh script first? I would have recommended doing that after the backup. The reason is that we have added a few retention policies and CQs (continuous queries) to InfluxDB since your last install (and the snapshot of your restore) and it is possible that the error is related to missing data from a CQ. You can try this from the Powerwall-Dashboard folder to create those missing CQs:

docker exec --tty influxdb sh -c "influx -import -path=/var/lib/influxdb/influxdb.sql"

If that doesn't help, can you see what the actual error might be?

docker logs grafana

Also, if you can, post a screenshot of the error.

[jaydkay]

Dear @jasonacox, thanks for your quick reply. No, I did not try the upgrade.sh script, I just set it up from scratch.

Since the command you mentioned above is also run during setup.sh, you mean I should run it again after restoring the backup? Do I need to stop the docker containers first or should the command be executed while the Powerwall-Dashboard is up?

[jasonacox]

Hi @jaydkay ! You are correct. When you restored the database, it wiped out all the changes setup.sh made to InfluxDB (the backup is effectively a snapshot of the database and config for InfluxDB).

You do not stop the containers. You need InfluxdDB to be running to accept the commands to create the retention policies and CQs:

docker exec --tty influxdb sh -c "influx -import -path=/var/lib/influxdb/influxdb.sql"

After you run that, it will take InfluxDB a few minutes to create the new measurements based on new incoming data from telegraf. But you should be able to refresh Grafana and see the data show up.

Can you post a screenshot of where you see the error?

[jaydkay]

Hi @jasonacox, just wanted to let you know that I have been succesfull in restoring the backup using your instructions above.

At first it didn't work, but then it occured to me that restoring the backup has also overwritten the file influxdb.sql with the older version from last july. After pulling the current version again and repeating the above command everything seems to be working again. Only the data of the last two days is missing now, of course. I made a backup of the influxdb-folder before restoring the backup (containing the data of the last two days) - do you know if there is an easy way to re-import this into the database to fill the gaps?

Below is the screenshot you requested of the error before repairing the influxdb installation.

Thanks a lot for your assistance!

[jasonacox]

That's great news @jaydkay !

In the future, I really recommend that you use the upgrade.sh tool to help save you the hassle. It's still good to do a backup to save as an emergency, but the upgrade.sh will update all the files but leave your data alone. It does occur to me that we don't have a comprehensive instruction set for someone who wanted to do an upgrade manually. TODO for me to add a "how to upgrade manually" section. 😁

do you know if there is an easy way to re-import this into the database to fill the gaps?

Yes!! Thanks to @mcbirse 's amazing work, you can use the Tesla import tool to fill in those gaps from the Tesla Cloud. Here are the instructions (and the tool is in the ./tools folder):

https://github.com/jasonacox/Powerwall-Dashboard/tree/main/tools/tesla-history#tesla-history-import-tool

It is a little invovled running it the first time to get your Tesla token, but if you follow the steps you will be able to fill in all the data missing from the time your purchased your Powerwall until now.

[jaydkay]

Hi @jasonacox, sorry for taking a while to answer. Meanwhile I've tried the tesla-history.py and I've been able to fill the gaps. Thanks a lot for pointing me in this direction!

I've also just updated to version 2.8.5 using upgrade.sh and that has worked flawlessly as well!

[jasonacox]

Great to hear! Thanks @jaydkay !