Compose-Plugin: Backup not working on compose files that include spaces + variables not interpreted

[mocaadmin]

I have two problems where I am not sure how to solve them:

1. Backup of Docker Plugin fails when there is a space character in the filename This is the example file structure in question: │ ├─ srv
│ │ ├─ dev-disk-by-uuid-4f8fcc..
│ │ ├─ dev-disk-by-uuid-b33d..
│ │ │ ├─ compose
│ │ │ │ ├─ Collabora
│ │ │ │ ├─ Heimdall
│ │ │ │ ├─ MQTT broker
│ │ │ │ ├─ Matrix Synapse Server
│ │ │ │ ├─ Network Setup
│ │ │ │ ├─ Nextcloud All in One
│ │ │ │ ├─ OMV Portainer
│ │ │ │ ├─ Traefik
│ │ │ │ └─ evcc
│ │ │ ├─ container

This is the logfile extract: [2023-11-05 00:00:18+0100] [composebackup] compose :: Nextcloud [2023-11-05 00:00:18+0100] [composebackup] Docker storage :: [2023-11-05 00:00:18+0100] [composebackup] Compose file path :: /srv/dev-disk-by-uuid-b33d.../compose [2023-11-05 00:00:18+0100] [composebackup] Backup path :: /srv/dev-disk-by-uuid-e3707../BACKUP/docker_backup
[2023-11-05 00:00:18+0100] [composebackup] Compose file '/srv/dev-disk-by-uuid-b33d../compose/Nextcloud/Nextcloud.yml' does not exist. Exiting... [2023-11-05 00:00:18+0100] [composebackup] compose :: evcc [2023-11-05 00:00:18+0100] [composebackup] Docker storage :: [2023-11-05 00:00:18+0100] [composebackup] Compose file path :: /srv/dev-disk-by-uuid-b33d../compose [2023-11-05 00:00:19+0100] [composebackup] Backup path :: /srv/dev-disk-by-uuid-e3707../BACKUP/docker_backup [2023-11-05 00:00:19+0100] [composebackup] Compose file :: /srv/dev-disk-by-uuid-b33d../compose/evcc/evcc.yml [2023-11-05 00:00:19+0100] [composebackup] Backup max size :: 1 GB

For evcc it works for Nextcloud all in one it doesn't. I assume that somewhere the path variable is cut due to the space character but I am not knowledgeable enough to propose a correction. Maybe the space should not be allowed in the filenames of the docker compose files?

2. My 2nd problem is that the backup plugin seems to ignore the global variables set for config folders & files Here's the extract from my global.env file cat /srv/dev-disk-by-uuid-b33d../compose/global.env
APPDATA_DIR=/srv/dev-disk-by-uuid-b33d796c-14b6-4f1d-91a5-f99ba80a5c53/container/appdata LOGS_DIR=/srv/dev-disk-by-uuid-b33d796c-14b6-4f1d-91a5-f99ba80a5c53/container/logs BASE_DIR=/srv/dev-disk-by-uuid-b33d796c-14b6-4f1d-91a5-f99ba80a5c53/compose

The backup logfile has the following error:

[2023-11-05 00:00:18+0100] [composebackup] Compose file path :: /srv/dev-disk-by-uuid-b33d../compose [2023-11-05 00:00:19+0100] [composebackup] Backup path :: /srv/dev-disk-by-uuid-e3707../BACKUP/docker_backup [2023-11-05 00:00:19+0100] [composebackup] Compose file :: /srv/dev-disk-by-uuid-b33d../compose/evcc/evcc.yml [2023-11-05 00:00:19+0100] [composebackup] Backup max size :: 1 GB [2023-11-05 00:00:22+0100] [composebackup] Volume :: $APPDATA_DIR/evcc/evcc.yaml:/etc/evcc.yaml [2023-11-05 00:00:22+0100] [composebackup] Backup :: $APPDATA_DIR/evcc/evcc.yaml:/etc/evcc.yaml [2023-11-05 00:00:22+0100] [composebackup] Path does not exist. Skipping... [2023-11-05 00:00:22+0100] [composebackup] Volume :: $APPDATA_DIR/evcc:/root/.evcc [2023-11-05 00:00:22+0100] [composebackup] Backup :: $APPDATA_DIR/evcc:/root/.evcc [2023-11-05 00:00:22+0100] [composebackup] Path does not exist. Skipping... [2023-11-05 00:00:25+0100] [composebackup] Done.

Any clue in which global variables it should be mentioned?

[ryecoaaron]

1. fixed but it might be a few days before I can release. https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-compose/commit/b09afb3e4a821f5014e7f0f17415af87ca4678cb
2. The backup does not parse the global environments file.

[TheIronboy]

hi, i’m running the latest version of omv with the compose plugin.

in the schedule section (of the compose plugin) i created a new schedule that updates all the images daily.

it pulls the images, but then, it doesn’t recreate the containers.

i made sure the containers are running and they are, and like it says, it should recreate the containers if they are running before the update starts.

@ryecoaaron can it be a problem with the plugin?

[ryecoaaron]

What version are you running? Your post is from after I released an update.

[TheIronboy]

i'm running the latest version (6.11.2)

[ryecoaaron]

Then it isn't detecting that any containers are up, then it doesn't do the "up -d". I would need more than one line of output to determine why that is. Are you using a space in the file name and that is why you added to this issue?

[TheIronboy]

no i'm not, the compose file and the container name are both without spaces, i just added to this issue becouse is the same "feature" (backup and upgrade). should i open another one?

edit: i opened the "omv-compose-update" file, and i ran the command you use to check the status of the containers.

when i run this: "docker compose ls --all --filter name="${compose}" --format json"

it works fine, but then when i add the remaining:

"| jq ".[0].Status"

it shows this:

for now i fixed it by removing completly the check and making it always recreate the containers, as all my containers are always up. but this is going to be wiped when an update of the plugin is released.

hopefully this is useful for you to find the problem.

[ryecoaaron]

Unfortunately, that doesn't help. For the command to work, the compose variable needs to be set. I just want you to run the script and post the output of it.

sudo omv-compose-update-multi

And yes, this should've been a separate issue. This one was specifically about the problem with a space in the name.

[TheIronboy]

sorry about that, however here is the output of the command.

edit: i think i found the problem, at line 30 of the omv-compose-update file you set in the "compose" variable the name of the .yml file.

then in at line 83 you use the same variable to check if it's running, but in order to check if it's running you need the real container name written inside the compose file.

example: in my log, the container name in openmediavault (and the .yml file) is "iCloud-Drive" but, the actual container name written inside the compose file is "icloud-drive"

i think this is the issue, if i'm wrong, i apologise.

[ryecoaaron]

docker compose ls only outputs the status of compose files not containers. But I just need to move the status detection like I did with the backup script - https://github.com/OpenMediaVault-Plugin-Developers/openmediavault-compose/commit/43af74423bf72ec63682615ff66e953af881096e

[ryecoaaron]

This should be fixed in 6.11.3. I was testing with a compose file called "space TEST 12-23" with a container named nginx7. No problems backing up, restoring, or updating.