New Feature Request - a non-manual backup and restore via the interface.

[Frono]

Would love to request a tools/button for backups, that would have a backup button potentially with options to backup (a. config b.birds database c.birdsong dir) into a compressed file. Also another function that would restore a backup file(s).

Reasoning. People where I am love these. I've been setting them up for non-technical friends, so just telling them to run import/export for the database and move files will not work for them (assuming the SD cards or pi's themselves go bad periodically).

My goal would be for me to be able to tell them to backup by clicking on the button from time to time and downloading the resultant compressed file, so when their pi dies. I can reinstall birdnet from this repo and upload their most recent backup file to make the system as it was when it was backed up last.

[alexbelgium]

Hi, currently you can backup files manually by using the Filebrowser addon as illustrated below. It's a bit more cumbersome but works ;-)

EDIT : the easiest way could be a cronjob to cp the relevant files to an external drive regularly ?

Globally you would propose that an automated way to do it is baked in... It could be quite feasible using shell scripts + an UI but still would take some time to get it right if we wanted a true non-technical proof system (especially as a true backup should be saved remotly such on google drive or onedrive...). I don't think there exists ready to use solutions that we could just plug in...

If I do a bit of shameless self-promotion, all these features of automatic backuping (including to google drive) are easily available in HomeAssistant (for which I've developed birdnet-pi as an addon) & docker systems (for which there is no publicly supported image I'm aware off) which could therefore be an easier solution... And would provide an easy interface for backup & management at the expense of increased resources usage.

[Frono]

I was thinking of shell script for ease. Where someone saves their backup would be their business. This way the Pi would be like any other network appliance or many applications. I'm not sure of the details, but I'd think it would be functionally be something like.

1. Click backup in UI
2. script starts
3. script shuts down services
4. script runs export of birds database
5. script creates archive with config file and database export
6. script restarts services
7. displays link to download backup file in the UI.

For a restore. To keep it simple I'd propose that the export file is imported for all records where the timedate stamp in the export file is not in the import file.

1. Click restore in UI
2. opens an upload file requester
3. user uploads the file
4. script shuts down services
5. script copies config file over current config (not sure about what import edits would be prudent)
6. script imports birds.db from export (can either truncate and restore, or add wherever timedate stamp doesn't exist from original export)
7. script resumes services

When my last SD card refused to write anymore, I tried to follow the instructions from Mcguire to back up and restore, the config file was easy, but you have to modify PHP files to get the export from the old database into the new database as, by default it only takes 2mb uploads. and my database was much larger. So to restore my old data would require writing the necessary sql, and running from command line. I'm thinking this may be beyond the person who has a pi, and can curl an install do to bird listening in their back yard. I've set up a few of these for bird lovers I know (as they have an unused pi they bought for the kids but lack the ability to do anything with it), and I know they want to look at historical data for year over year comparisons. However, I can guarantee they lack the skills to perform the current steps for backup/restore themselves, and probably would not be able to develop them due to lack of skill-set, and often lifespan... :)

I have a few pi's around and can help anyone willing to give it a go with testing...

[krummrey]

I've lost my first database with 1,5 years of data, because the SD-Card got totally corrupted. So having a backup on the card seems less tan ideal to me. Options to copy the .zip file to a cloud service (Dropbox, OneDrive, iDrive, …) automatically would be something I'd love to see.

[alexbelgium]

Hi, I do see the interest but this will be something complex to set-up and I'll soon have much less time to support... Perhaps I'll try to start but it will likely be a team effort if we manage to achieve that ! It would be great if a pre-existing solution existing that we could just plug-in with an iframe as for the Filebrowser or Systeminfo... Any ideas on a pragmatic solution would be welcome :-)

Edit : I do believe however a cron job that copies important file on a usb drive plugged into the rpi might be a very good easy mitigation though...

Edit 2 : this tool seems promising https://github.com/sebastianfeldmann/phpbu ; but it has no webui...

[craigusus]

I was thinking about creating a cron job to run a script too. I asked my new friend phind and he/she/it recommended rclone which has the means to connect to loads of different types of file storage (Google Drive, FTP, SSH, SMB, local, Dropbox, iDrive, anything S3 compliant etc). There is also (albeit experimental) a Web GUI.

[alexbelgium]

Excellent idea and indeed rclone seems great ! It's a nice coincidence that the web gui is just getting an update after 3 years ; but perhaps it would be more "user friendly" if we could find a way to run shell commands from a php page to simply execute. According to your friend that I just got acquainted with (quite nice indeed), it should be possible to manage a few storage locations in a headless way if the users provide their credentials...

We perhaps could split the work in two : first create the "restore" feature that would allow to upload a zip archive containing the relevant files, then put them at the right place, and second create the "backup" feature that would be a bit more complex.

Edit : here is the list of files that could be useful to backup (= that I have selected to make remanent in my docker-based addon) - I think the list from mcguire's backup & restore instruction was much more shorter for files in "$HOME/BirdNET-Pi/birdnet.conf" "$HOME/BirdNET-Pi/scripts/birds.db" "$HOME/BirdNET-Pi/scripts/disk_check_exclude.txt" "$HOME/BirdNET-Pi/apprise.txt" "$HOME/BirdNET-Pi/exclude_species_list.txt" "$HOME/BirdNET-Pi/include_species_list.txt" "$HOME/BirdNET-Pi/IdentifiedSoFar.txt"; do and folders $HOME/BirdSongs/By_Date $HOME/BirdSongs/Charts ; there could be the /etc/caddy/Caddyfile too if it was customized

[Nachtzuster]

I have an (old) import script for importing data from one BirdNET-Pi to another. the idea is that you have the 'old' sd card mounted (via a sd->usb adapter) on the new system and then run the script https://github.com/Nachtzuster/BirdNET-Pi/blob/import_data/scripts/import_data.sh

This might be adapted for a proper backup script

[Nachtzuster]

I've reworked the import_data script: see https://github.com/Nachtzuster/BirdNET-Pi/pull/119 Have a look?

[alexbelgium]

Thanks i'll look tomorrow !