How to setup rsync

[Bumpay]

As far as I know rsync should only do one full backup and then have only backups of the changes. But why do I get only full backups all the time? Did I configure something wrong?

compose.yml

services:

  mc:
    container_name: mc-vanilla-plus
    image: itzg/minecraft-server
    stdin_open: true
    ports:
      - "25565:25565"
      - "25575:25575"
    volumes:
      - ./data:/data
    depends_on:
      restore_backup:
        condition: service_completed_successfully
    environment:
      EULA: true
      MEMORY: "6G"
      MOD_PLATFORM: "MODRINTH"
      MODRINTH_MODPACK: "cutika-vanilla+-server.mrpack"
      RCON_PASSWORD: "####"
    restart: unless-stopped

  restore_backup:
    container_name: restore-vanilla-plus
    image: itzg/mc-backup
    restart: "no"
    entrypoint: restore-rsync-backup
    volumes:
      - ./data:/data
      - ./backups:/backups:ro

  backups:
    container_name: backups-vanilla-plus
    image: itzg/mc-backup
    depends_on:
      mc:
        condition: service_healthy
    environment:
      BACKUP_INTERVAL: "2h"
      BACKUP_METHOD: "rsync"
      RCON_HOST: mc
      INITIAL_DELAY: 0
    volumes:
      - ./data:/data:ro
      - ./backups:/backups

[itzg]

It's implemented using rsync --link-dest such that it does incremental backups but utilizes hard links to give the behavior of each backup being full contained.

[Bumpay]

Ah alright, I was looking at the directories through FileZilla and thought, they are all backing up redundant parts of the world. But running du -sh * shows, that the backups are actually really small in size. Thanks for pointing that out.

Here is the result of the du -sh * command:

16G     world-20240703-080010
3.8M    world-20240703-080101
3.8M    world-20240703-081010
3.8M    world-20240703-082010
3.8M    world-20240703-082418
3.8M    world-20240703-091909
81M     world-20240703-111410
237M    world-20240703-130911
105M    world-20240703-140912
106M    world-20240703-150914
142M    world-20240703-160915
435M    world-20240703-170916

[Bumpay]

By the way, how does it prune the backups with the rsync backup method?

[itzg]

It uses a find by modified time and does rm -r on those directories

https://github.com/itzg/docker-mc-backup/blob/3d04028e75df576fc1bdf83a177c83fe30c58a9d/scripts/opt/backup-loop.sh#L323

Being hard links it works out to prune away each top level backup directory.

[Bumpay]

Ah okay thanks, so to confirm I understand that correct. As long as there is a backup that hard links to the top level backup the top level backup will not be pruned?

Then, when are top level backups created? Is it just once in the beginning? And what would happen to the linked backups if the top level backup is being removed?

Sorry if I don't understand the (hard) linking, I'm fairly new to working with Linux.

[itzg]

Understandable, hard links don't get talked about much but they're a pretty powerful feature in this scenarios. This description is pretty good but was hoping to find something that explained more concisely

https://www.geeksforgeeks.org/soft-hard-links-unixlinux/