geerlingguy / internet-pi

Raspberry Pi config for all things Internet.
MIT License
4.3k stars 449 forks source link

Create Backup guide #194

Closed geerlingguy closed 1 month ago

geerlingguy commented 3 years ago

See #7.

For backup for Pi-hole at least, in the GUI you can go to Settings > Teleporter and click 'Backup'. To automate it through the console, you can run pihole -a -t (can I integrate it with something like my geerlingguy.backup role?).

If I run docker-compose exec pihole /bin/bash pihole -a -t it exports a file inside the container. docker-compose cp is not a supported command yet (see https://github.com/docker/compose/issues/3593), so will have to figure out the best way of scripting this... maybe something like:

  1. Create a /backup directory inside the container (mkdir -p /backup).
  2. Run the pihole -a -t command inside the /backup directory.
  3. Cat the contents of /backup/[latest-file-with-extension].tar.gz and pipe them to file on host.
  4. Delete contents of /backup directory (rm -rf /backup/*).

It would be convenient for the pihole -a -t command to have an option to output the archive to shell instead of into a file, that would save the whole file-based annoying process.

geerlingguy commented 3 years ago

I would also like to make it so you could backup the Prometheus data as well, from the configured Docker volume.

Jatidude commented 3 years ago

You could mount that /backup folder as a volume in your docker compose, so the host has access to the files there without needing your step 3.

Your script could docker-compose exec pihole -a -t into the /backup volume mounted and then you can rsync it off somewhere or whatever other backup method that you want.

shawnsBrain commented 2 years ago

You could query out the data from grafana externally using the built-in API.

replace the IP address (here as 192.168.88.185) with the IP of your Pi. Replace the start and end seconds, and "step" is the seconds resolution. It returns a JSON string.

download query: http://192.168.88.185:3030/api/datasources/proxy/12/api/v1/query_range?query=speedtest_download_bits_per_second{}&start=1630847394&end=1631082288&step=300

upload query http://192.168.88.185:3030/api/datasources/proxy/12/api/v1/query_range?query=speedtest_upload_bits_per_second{}&start=1630847394&end=1631082288&step=300

ping query http://192.168.88.185:3030/api/datasources/proxy/12/api/v1/query_range?query=speedtest_ping_latency_milliseconds&&start=1632928800&&end=1633533600&&step=600

jrwagz commented 2 years ago

Another simple example, is just to store all data that you want to easily backup into a volume, and then you can easily script the export of those volumes through another temporary container that you spin up just for the purposes of data-exporting.

https://docs.docker.com/storage/volumes/#backup-restore-or-migrate-data-volumes

Beutelwolf commented 2 years ago

Another way would be to map all volumes you want to backup to your host and then use a dedicated backup client container like duplicati

https://hub.docker.com/r/duplicati/duplicati/

to manage your backup.

stale[bot] commented 2 years ago

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!

Please read this blog post to see the reasons why I mark issues as stale.

stale[bot] commented 1 year ago

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!

Please read this blog post to see the reasons why I mark issues as stale.

stale[bot] commented 1 year ago

This issue has been closed due to inactivity. If you feel this is in error, please reopen the issue or file a new issue with the relevant details.

dracostheblack commented 1 year ago

@geerlingguy it looks like Prometheus has the ability to create snapshots for backup but you need the admin api enabled https://prometheus.io/docs/prometheus/latest/querying/api/#snapshot. How can we do that through the configuration?

Was using this to back up grafana to s3 https://github.com/ysde/grafana-backup-tool, just need the data now to complete it i think.

github-actions[bot] commented 3 months ago

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!

Please read this blog post to see the reasons why I mark issues as stale.

github-actions[bot] commented 1 month ago

This issue has been closed due to inactivity. If you feel this is in error, please reopen the issue or file a new issue with the relevant details.