Open CardcaptorRLH85 opened 3 years ago
Hi @CardcaptorRLH85, sorry to hear about your issues. Luckily you were able to restore the Docker engine to its original state. The log file and configuration all look normal to me, right until step 9 that is. A few thoughts and questions, if you're up for additional investigation.
sudo synoservicectl --start pkgctl-Docker
)?I'd be glad to help out.
{
"data-root" : "/var/packages/Docker/target/docker",
"log-driver" : "db",
"registry-mirrors" : [],
"storage-driver" : "btrfs"
}
When I tried shutting down all of my containers first just now, it worked. I probably should have tried that but it just never crossed my mind.
EDIT: It seems like there's a different issue now, unfortunately. When I try to start a container I see this message:
CardcaptorRLH85@Rin-san:~/synology-docker$ sudo docker start Portainer-CE
Password:
Error response from daemon: failed to initialize logging driver: failed to get logging factory: logger: no log driver named 'db' is registered: error looking up logging plugin db: plugin "db" not found
Error: failed to start containers: Portainer-CE
CardcaptorRLH85@Rin-san:~/synology-docker$ docker rename Portainer-CE portainer
CardcaptorRLH85@Rin-san:~/synology-docker$ sudo docker run -d -p 8000:8000 -p 9000:9000 --name=Portainer-CE --restart=always -v /var/run/docker.sock:/var/run/docker.sock -v /volume1/docker/Portainer-CE:/data portainer/portainer-ce
docker: Error response from daemon: Failed to create btrfs snapshot: inappropriate ioctl for device.
See 'docker run --help'.
Without looking into it I'm guessing that the db logging driver is a Synology-specific thing (after looking for just a couple of minutes I didn't see it mentioned in Docker's documentation anywhere) so, I'd understand if it doesn't work after an update. As my terminal output demonstrates, I then decided to rename the old container and recreate it from the command line. Unfortunately, I then got the docker: Error response from daemon: Failed to create btrfs snapshot: inappropriate ioctl for device.
error.
Secondly, I've noticed that after this update all of the environment variables on my containers have been removed. I don't know what happened, they're just gone.
Thanks for the follow up @CardcaptorRLH85.
I'm using Btrfs.
The btrfs
support seems to questionable (see #22). My NAS uses ext4
, so I haven't found a way to test this myself. Does below configuration work on your NAS? This is a fix that I'll address in #40.
{
"data-root" : "/var/packages/Docker/target/docker",
"log-driver" : "json-file",
"registry-mirrors" : [],
"storage-driver" : "btrfs"
}
Without looking into it I'm guessing that the db logging driver is a Synology-specific thing
The db
logging driver is indeed a custom driver written by Synology. They use it as input for their GUI. I haven't found the source myself, reason why this repository replaces db
with the default json-file
driver. As an unwanted side-effect, Synology displays the wrong up time in the GUI unfortunately.
Secondly, I've noticed that after this update all of the environment variables on my containers have been removed.
I have heard about similar issues from other users. I haven't found a way to recreate the issue yet. For now the only thing I can recommend is to use Docker Compose instead. It allows you to script the entire container initiation, including environment variables. Hopefully it won't cost you to much effort to recreate your containers. I'll add this as a known issue too.
I'd just finished reading the Known Issues and was about to edit my comment again before you replied. Unfortunately, that change to my config doesn't fix things. Since this makes the third time that I'll be editing the variables for ~2 dozen containers in the last few days, I'll certainly take your Docker Compose advice after restoring to the Synology version of Docker again.
The storage driver seems to be a tough one to fix. Docker's documentation provides some additional clues. If you have the time and patience, you could try alternative storage drivers instead. See below table for an overview.
Storage driver | Supported backing filesystems |
---|---|
overlay2, overlay | xfs with ftype=1, ext4 |
fuse-overlayfs | any filesystem |
aufs | xfs, ext4 |
devicemapper | direct-lvm |
btrfs | btrfs |
zfs | zfs |
vfs | any filesystem |
I haven't tested this myself, but I'd be curious to know if any one of these storage drivers help. A warning from Docker: | Important: When you change the storage driver, any existing images and containers become inaccessible. This is because their layers cannot be used by the new storage driver. If you revert your changes, you can access the old images and containers again, but any that you pulled or created using the new driver are then inaccessible. |
---|
I got a question, I'm trying to change my storage driver to overlay2, my backend is already a ext4 volume. The docker daemon fail to start but i can't figure out where the daemon log are located. The /var/log/Docker
is empty.
Context: I'm to run the k3s agent on my synology so that it can act as a node for storage but it require an overlay storage driver.
Does /var/log/upstart/pkg-Docker-dockerd.log
provide any clues?
Oh thanks! Just checked, now I got something to investigate :
2021-04-07T18:33:08-0400 ERRO[2021-04-07T18:33:08.496803188-04:00] failed to mount overlay: no such device storage-driver=overlay2
Same thing with overlay
. After checking /proc/filesystems
I realized the DS918+ doesn't seems to support overlayfs. Checked the /lib/modules
and ran modprobe overlay
but nothing there.
Kernel version is 4.4.59+
(DSM 6.2.3) which should include overlay from my understanding but they might have removed it from their distro.
I could try to install fuse-overlayfs.
It's a pity that Synology does not support the overlay driver. They tend to heavily modify their distribution, so it doesn't come as a surprise.
I came across this article on Medium from Kristofer Lundgren, who used Docker in Docker instead of upgrading the Docker daemon. Might be worth investigating?
I also have this issue (doesn't start), but this is the error I see in: /var/log/upstart/pkg-Docker-dockerd.log
2023-02-25T08:28:34-0500 ERRO[2023-02-25T08:28:34.505536938-05:00] [graphdriver] prior storage driver aufs is deprecated and will be removed in a future release; update the the daemon configuration and explicitly choose this storage driver to continue using it; visit https://docs.docker .com/go/storage-driver/ for more information 2023-02-25T08:28:33-0500 INFO[2023-02-25T08:28:33.703362207-05:00] [core] [Channel #4] Channel Connectivity change to SHUTDOWN module =grpc 2023-02-25T08:28:33-0500 INFO[2023-02-25T08:28:33.704084712-05:00] [core] [Channel #4 SubChannel #5] Subchannel Connectivity change to SHUTD OWN module=grpc 2023-02-25T08:28:33-0500 INFO[2023-02-25T08:28:33.704700088-05:00] [core] [Channel #4 SubChannel #5] Subchannel deleted module=grpc 2023-02-25T08:28:33-0500 INFO[2023-02-25T08:28:33.705372381-05:00] [core] [Channel #4] Channel deleted module=grpc 2023-02-25T08:28:33-0500 INFO[2023-02-25T08:28:33.706815321-05:00] stopping event stream following graceful shutdown error="context
This is on an old DS712+ running DSM 6.2.4-25556 Update 6 So it looks like if the current storage driver is 'aufs' then for newer docker versions this either need to be explicit to 'overlay2'?
I may try this but a little nervous based on others attempts at changing storage drivers.
I also have this issue on a DS1815+ with DSM7.1.1.
bokkoman@FLOPPYDISK:~$ sudo systemctl start pkgctl-Docker
Job for pkgctl-Docker.service failed. See "systemctl status pkgctl-Docker.service" and "journalctl -xe" for details.
bokkoman@FLOPPYDISK:~$ systemctl status pkgctl-Docker.service
● pkgctl-Docker.service - Docker's service unit
Loaded: loaded (/usr/local/lib/systemd/system/pkgctl-Docker.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Tue 2024-06-11 13:10:54 CEST; 28s ago
Process: 18451 ExecStart=/bin/bash -c /usr/syno/sbin/synopkgctl start $SELF && /bin/touch /var/packages/$SELF/enabled (code=exited, status=1/FAILURE)
Main PID: 18451 (code=exited, status=1/FAILURE)
Describe the bug Exactly as it says in the title, the Docker service doesn't start after being updated with this script. Fortunately, everything still works if I restore the binaries from the backup but, the entire reason for me to use this script was to get a more up-to-date version of Docker due to the known issue of not being able to update environment variables with the version currently supplied by Synology.
Here is the specific error message:
To reproduce I ran the following command:
sudo ./syno_docker_update.sh update
and it failed at "Step 9 from 10: Starting Docker service".Expected behavior I expected Docker to simply start running again after the update.
Log file
Docker daemon configuration
Additional context As I was going back through the steps to collect the data for this bug report, I decided to try installing Docker version 19.03.14 (currently the latest version of the 19.x branch) instead of the latest version (as of now 20.10.2). Unfortunately, that too failed in the same manner. So, I finally tried an upgrade to the most recent version of the 18.x branch, 18.09.9 which is only one revision (and just under two months) newer than the eighteen-month-old 18.09.8 version that Synology currently ships. However, that also failed in the same manner.