Add ability to backup composed configuration that are using [included ...cfg] files.

[mmemetea]

Is your feature request related to a problem? Please describe.

My current configuration is exploded in several smaller and easier-to-manage sub-directories such as this:

#########################
### MAIN CONFIG FILES ###
#########################
[include config/machine.cfg] # Keep this included!
[include config/variables.cfg] # Keep this included!

### MCUs configuration ---------------------------------------------
[include config/mcus/SKR1.4_x2_V2.4.cfg]
[include config/mcus/rpi.cfg]
# [include config/mcus/spider_v1.cfg]
# [include config/mcus/spider_v2.cfg]
# [include config/mcus/octopus.cfg]
# [include config/mcus/canboard_SHT36-42.cfg]
# ------------------------------------------------------------------

###########################
### HARDWARE COMPONENTS ###
###########################

### XY axis configuration ------------------------------------------
[include config/hardware/XY/0.9d_xy.cfg]
# [include config/hardware/XY/1.8d_xy.cfg]
# ------------------------------------------------------------------

In my particular case, the complete list of files would be:

printer_data/config/config/hardware/displays/mini12864.cfg printer_data/config/config/hardware/extruder/galileo.cfg printer_data/config/config/hardware/fans/chamber_fan.cfg printer_data/config/config/hardware/fans/controller_fan.cfg printer_data/config/config/hardware/fans/hotend_fan.cfg printer_data/config/config/hardware/fans/part_fan.cfg printer_data/config/config/hardware/heated_bed.cfg printer_data/config/config/hardware/lights/fcob_white.cfg printer_data/config/config/hardware/lights/neopixel_fystec12864.cfg printer_data/config/config/hardware/probe.cfg printer_data/config/config/hardware/temperature_sensors/chamber_temp.cfg printer_data/config/config/hardware/temperature_sensors/rpi_temp.cfg printer_data/config/config/hardware/XY/0.9d_xy.cfg printer_data/config/config/hardware/Z/V2.4_1.8d_base_z.cfg printer_data/config/config/machine.cfg printer_data/config/config/mcus/rpi.cfg printer_data/config/config/mcus/SKR1.4_x2_V2.4.cfg printer_data/config/config/software/bed_mesh.cfg printer_data/config/config/software/firmware_rectraction.cfg printer_data/config/config/software/input_shaper.cfg printer_data/config/config/software/tilting/qgl.cfg printer_data/config/config/software/z_calibration.cfg printer_data/config/config/variables.cfg

But ideally, this should just get all the files below config ending in .cfg. (results from find home/pi/printer_data/config/ -type f -name '*.cfg')

Describe the solution you'd like

I want the current script to be able to fetch these sub-directories.

Describe alternatives you've considered

None so far. I tried to modify the .env file to include those but without success.

Additional information

No response

[Tylerjet]

How are your paths setup in the .env file? For example printer_data/config/* would recursively grab all the files and folders that are not-symbolic links and copy them to the backup folder.

[ManeLippert]

Dear All,

if I understand problem right the backup script does not upload subdirectories. I had the same issue as well and found the underlying issue.

In commit 326db08 @Tylerjet replaced line 105

cp -r --parents "$file" "$backup_path/"

with

rsync -R "$file" "$backup_path/"

this will sync only the files in ~/printer_data/config/ and skipping every subdirectories.

Output:

Klipper-backup is up to date

hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint: 
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint: 
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
From https://github.com/repository
 * branch            main       -> FETCH_HEAD
Already up to date.
skipping directory printer_data/config/macros
skipping directory printer_data/config/moonraker
skipping directory printer_data/config/printer

An easy fix for this issue is to add the -r flag into the rsync command

rsync -rR "$file" "$backup_path/"

Additionally one could only add -r flag and removes -R flag which changes the folder structure in the backup repo from

printer_data/config/
.gitignore
README.md

to

printer.cfg
moonraker.cfg
...
.gitignore
README.md

Hope this helps to fix the issue.

PS: -a includes also symlinks because rsyncskips them by default.

[mmemetea]

I think @ManeLippert has found the reason why it doesn't work as intended. The subdirectories aren't symlinked, so it should work, but the recent change is preventing this. My installation is quite recent, so I can't know if it would have worked before.

[mmemetea]

Just tried @ManeLippert suggestion. It works beautifully. Thanks.

[ManeLippert]

@mmemetea No problem. Glad I could help.

[Tylerjet]

ahh indeed we forgot to keep -r for recursive in thanks for the catch will place a pr here in a few moments.

as for using -a the reason for not backing up symlinks i noted here (tl:dr is sym links are read only files since they are linked to files is repositories eg.) KAMP. When KAMP is updated all those changes would be gone and reverted) https://github.com/Staubgeborener/klipper-backup/issues/69#issuecomment-1965839873

And -R is used so that the folder structure it came from is backed up, this is for future intentions of a restore function so we want to have that path in the backup.

I also think its a bit nicer on the eyes if you choose to backup multiple folders you can see where all the files were backed up from, so a file at printer_data/config/printer.cfg is also at printer_data/config/printer.cfg in the repository.

[Tylerjet]

@mmemetea @ManeLippert You can now update to latest and -r is included in rsync. Thanks!