Docker Swarm support

[PabloRapidScale]

Is it possible to run this on docker swarm? I tried deploying but it doesn't seem to generate any actual files. Below is my config.

  backup:
    image: ghcr.io/realorangeone/db-auto-backup:latest
    container_name: backup
    environment:
      - INCLUDE_LOGS=true
      - COMPRESSION=gzip
    restart: unless-stopped
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - backup_data:/var/backups
    deploy:
      restart_policy:
        condition: on-failure
      mode: replicated
      replicas: 1
      placement:
        constraints: [node.role == manager]

networks:
  cloud-public:
    external: true

volumes:
  backup_data:
    driver: local
    driver_opts:
       o: bind
       type: none
       device: /volume1/docker/db/backups

[RealOrangeOne]

Theoretically it just talks to the docker socket, so as long as it can find containers using that, it should work. Do the logs tell you anything?

[PabloRapidScale]

They do not: Running backup with schedule '0 0 *'.

No further entries.

[RealOrangeOne]

How long is the container running? Is it definitely running at midnight when the backup is due to run?

[RealOrangeOne]

I've pushed an image update which adds a little more logging to hopefully help understand what's going on. Pull it and see what it says next time.

[PabloRapidScale]

Thanks, I pulled the updated image and ran it with $SCHEDULE as empty:

Starting backup... Found 16 containers. Backup of 0 containers complete in 0.09 seconds.

Running image: ghcr.io/realorangeone/db-auto-backup:latest@sha256:9adef19113dd38844115d97524771856237803d7891120db3dfd27bd5627a9c6

Note I do have at least one of the databases mysql running on the same node "Manager" as the db auto backup container so while it might not be able to access machines in another node in the swarm it should be able to reach this one on the same machine.

Below is the configuration for the mysql server:

mysql:
    image: ghcr.io/linuxserver/mariadb:latest
    container_name: mysql
    environment:
      - PUID=${PUID}
      - PGID=${PGID}
      - TZ=${TIMEZONE}
      - LOG_LEVEL=${LOG_LEVEL}
      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD?Variable not set}
      - MYSQL_DATABASE=${MYSQL_DB?Variable not set}
      - MYSQL_USER=${MYSQL_USER?Variable not set}
      - MYSQL_PASSWORD=${MYSQL_PASSWORD?Variable not set}
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - mysql_data:/config
    ports:
      - 3306:3306
    restart: unless-stopped
    networks:
      - cloud-public
    deploy:
      restart_policy:
        condition: on-failure
      mode: replicated
      replicas: 1
      placement:
        constraints: [node.role == manager]

[RealOrangeOne]

If it can't access it, there should be an error. It found 16 containers, but apparently didn't believe it could back any of them up. Does 16 total containers sound about right across your cluster, or could that just be on the same node? Even then, it ought to work.

I suspect adding more debug logging would be helpful. I'll see about adding some extra options for that at some point soon.

[PabloRapidScale]

That's just from the one node that it's sitting on:

$ docker ps | wc -l
16

Total containers in the cluster is about 47. As I mentioned there is at least one container that should qualify sitting on the same node running mariadb.

[PabloRapidScale]

I did some testing and issue for me at least appears to be with the get_container_names function. Problem is that it's not able to parse out the BACKUP_PROVIDERS variable from my containers if I use a full URL to define the image or a simpler name.

Works: image: adguard/adguardhome:latest

Doesn't work: image: postgres:15 image: ghcr.io/linuxserver/mariadb:latest

Made some adjustments and was able to at least find the two database containers running in my node:

def get_container_names(container: Container) -> Iterable[str]:
    names = set()

    # Check if the container has image tags
    if container.image.tags:
        for tag in container.image.tags:
            print(f"Tag found: {tag}")
            # Strip out registry, tag, and digest from the image name
            image_name = tag.split(":")[0].split("@")[0]  # Get just the image name without tag and digest
            image_name = image_name.split("/")[-1]  # Strip any registry or path prefixes
            names.add(image_name)

    # Fallback to image name in the container attributes
    if not names and container.attrs.get("Config", {}).get("Image"):
        print(f"No tags found, using image name: {container.attrs['Config']['Image']}")
        image_name = container.attrs["Config"]["Image"].split(":")[0].split("@")[0]
        image_name = image_name.split("/")[-1]  # Strip any registry or path prefixes
        names.add(image_name)

    return names

Running it manually does pick up two containers that fit the profile:

# python /var/backups/db_backup_test.py 
Starting backup...
Found 16 containers.
No tags found, using image name: ghcr.io/realorangeone/db-auto-backup:latest@sha256:9adef19113dd38844115d97524771856237803d7891120db3dfd27bd5627a9c6
...
No tags found, using image name: ghcr.io/linuxserver/mariadb:latest@sha256:a2bd0d13da8294acd4e264fdbe6abd513ef6443391ef9756a55dc7d0a8039060
db_mysql.1.hyxhn90o65apcl1w70wjoc2gt: 2.37MB [00:00, 10.9MB/s]
...
No tags found, using image name: postgres:15@sha256:99981723cfb0b44e09a7bac386eafde3c151bf427910d953e61d0f0ed39f596b
recipe_mealie_db.1.wxoirwuy9g5tb9v60p4gxvtac: 158kB [00:00, 237kB/s]  
Tag found: adguard/adguardhome:latest
...
Backup of 2 containers complete in 4.59 seconds.

[RealOrangeOne]

I'm surprised an image can exist but not be tagged with anything. Is that something Swarm does? I'd gladly welcome a PR adding support for reading the container's config on top of its tags.

[PabloRapidScale]

Some testing does seem to reveal that the way swarm handles tags is a little different from a standalone docker node. Containers managed by Docker Swarm frequently lack tags. Instead, they expose the raw image name with digests (e.g., ghcr.io/realorangeone/db-auto-backup:latest@sha256:9adef191...). While standalone docker will have proper tags. This is likely due to docker swarm managing the container as a service.

Standalone docker would still use container.image.tags for the parsing while seems using container.attrs['Config']['Image'] as a fallback. That still doesn't account for services in the swarm vs. just containers on the same node that match the lookup. I'll fiddle around with it and submit a PR if I come up with something that works.

Thanks!