Docker not properly working on DL4100

[roblandry]

Platform DL4100

Application Docker

Describe the bug Install works, but unable to mount directories. I just did a full factory reset, and reinstalled docker from scratch. I ensured the old docker was not running before installing via gui. I also removed the old _docker folder.

# /usr/sbin/docker_daemon.sh shutdown
# rm -rf /shares/Volume_1/Nas_Prog/_docker

Docker log: https://gist.github.com/roblandry/5e7a07c71f5a07c84ad97cc4a5a733f5

Info:

# uname -a
Linux WDMyCloudDL4100 3.10.38 #1 SMP Thu Aug 29 20:49:31 PDT 2019 Build-gitcdc15cb x86_64 GNU/Linux

# ./daemon.sh status && echo OK
OK

# ./daemon.sh issetup && echo OK
Docker setup NOK

# docker ps -a
CONTAINER ID        IMAGE                 COMMAND             CREATED             STATUS              PORTS                    NAMES
1fc5f4e7712b        portainer/portainer   "/portainer"        7 minutes ago       Created             0.0.0.0:9000->9000/tcp   frosty_hofstadter

# docker --version
Docker version 19.03.5, build 633a0ea838

# which docker
/sbin/docker

# ls -l /var/lib
lrwxrwxrwx    1 root     root            33 Jan 25 10:42 docker -> /shares/Volume_1/Nas_Prog/_docker
drwxr-xr-x    2 root     root             0 Jan 25 10:42 dpkg
drwxr-xr-x    2 root     root             0 Jan 25 10:42 samba

# ls -l ../_docker
drwx------    2 root     root          4096 Jan 25 10:35 builder
drwx--x--x    4 root     root          4096 Jan 25 10:35 buildkit
drwx------    3 root     root          4096 Jan 25 10:35 containerd
drwx------    3 root     root          4096 Jan 25 10:36 containers
-rw-r--r--    1 root     root        102007 Jan 25 10:56 docker.log
drwx------    3 root     root          4096 Jan 25 10:35 image
drwxr-x---    3 root     root          4096 Jan 25 10:35 network
drwx------    4 root     root          4096 Jan 25 10:35 plugins
drwx------    2 root     root          4096 Jan 25 10:42 runtimes
drwx------    2 root     root          4096 Jan 25 10:35 swarm
drwx------    2 root     root          4096 Jan 25 10:42 tmp
drwx------    2 root     root          4096 Jan 25 10:35 trust
drwx------    3 root     root          4096 Jan 25 10:42 vfs
drwx------    2 root     root          4096 Jan 25 10:35 volumes

# sh stop.sh .
DOCKER stop: stop daemon
Remaining mounts:

# sh start.sh .
DOCKER START: setup daemon
DOCKER START: start daemon
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

# docker start frosty_hofstadter
Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \\\"/var/run/docker.sock\\\" to rootfs \\\"/mnt/HD/HD_a2/Nas_Prog/_docker/vfs/dir/ccda66261c444d3c201924b0976525c4dcccb35678e3a8894e1160deeccdb097\\\" at \\\"/mnt/HD/HD_a2/Nas_Prog/_docker/vfs/dir/ccda66261c444d3c201924b0976525c4dcccb35678e3a8894e1160deeccdb097/var/run/docker.sock\\\" caused \\\"invalid argument\\\"\"": unknown
Error: failed to start containers: frosty_hofstadter

# docker info
Client:
 Debug Mode: false

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 1
 Server Version: 19.03.5
 Storage Driver: vfs
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: b34a5c8af56e510852c35414db4c1f4fa6172339
 runc version: 3e425f80a8c931f88e6d94a8c831b9d5aa481657
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 3.10.38
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 5.819GiB
 Name: WDMyCloudDL4100
 ID: PRW4:LG6G:4ZRW:DL7W:W6AZ:OJRN:XWJZ:HZ44:5B35:ZN7F:7AGV:Y73W
 Docker Root Dir: /mnt/HD/HD_a2/Nas_Prog/_docker
 Debug Mode: true
  File Descriptors: 20
  Goroutines: 40
  System Time: 2020-01-25T11:02:10.993388975-08:00
  EventsListeners: 0
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

WARNING: No kernel memory limit support
WARNING: No kernel memory TCP limit support

Additional:

# cat /proc/mounts | grep var/lib/docker

# cat /proc/mounts 
rootfs / rootfs rw 0 0
sysfs /sys sysfs rw,relatime 0 0
mdev /dev tmpfs rw,relatime 0 0
proc /proc proc rw,relatime 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
squash /usr/local/tmp ramfs rw,relatime,size=101m 0 0
/dev/loop0 /usr/local/modules squashfs ro,relatime 0 0
tmpfs /usr/local/config tmpfs rw,relatime,size=8192k,nr_inodes=0 0 0
tmpfs /mnt tmpfs rw,nosuid,nodev,noexec,relatime,size=1024k,nr_inodes=0 0 0
tmpfs /var/log tmpfs rw,nosuid,nodev,noexec,relatime,size=40960k,nr_inodes=0 0 0
tmpfs /tmp tmpfs rw,nosuid,nodev,noexec,relatime,size=102400k,nr_inodes=20000 0 0
tmpfs /tmp/wdnas_snap_config tmpfs rw,relatime,size=8192k,nr_inodes=0 0 0
/dev/sda4 /mnt/HD_a4 ext4 rw,nosuid,nodev,noexec,noatime,nodiratime,nobarrier,quota,usrquota,grpquota,data=ordered 0 0
/dev/sda2 /mnt/HD/HD_a2 ext4 rw,noatime,nodiratime,nobarrier,quota,usrquota,grpquota,stripe=128,data=ordered 0 0
cgroup /sys/fs/cgroup tmpfs rw,relatime,mode=755 0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/cpu cgroup rw,relatime,cpu 0 0
cgroup /sys/fs/cgroup/cpuacct cgroup rw,relatime,cpuacct 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,relatime,memory 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,relatime,devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,relatime,freezer 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,relatime,blkio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup rw,relatime,perf_event 0 0
/dev/sda2 /mnt/HD/HD_a2/Nas_Prog/_docker ext4 rw,noatime,nodiratime,nobarrier,quota,usrquota,grpquota,stripe=128,data=ordered 0 0

[roblandry]

It appears the same issue happens when trying to use portainer with the default docker installed.

# docker --version
Docker version 1.7.0, build 0baf609

# docker info
Containers: 0
Images: 0
Storage Driver: devicemapper
 Pool Name: docker-8:2-34603014-pool
 Pool Blocksize: 65.54 kB
 Backing Filesystem: extfs
 Data file: /dev/loop1
 Metadata file: /dev/loop2
 Data Space Used: 305.7 MB
 Data Space Total: 107.4 GB
 Data Space Available: 107.1 GB
 Metadata Space Used: 729.1 kB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.147 GB
 Udev Sync Supported: false
 Deferred Removal Enabled: false
 Data loop file: /mnt/HD/HD_a2/Nas_Prog/_docker/devicemapper/devicemapper/data
 Metadata loop file: /mnt/HD/HD_a2/Nas_Prog/_docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.82-git (2013-10-04)
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 3.10.38
Operating System: <unknown>
CPUs: 2
Total Memory: 5.819 GiB
Name: WDMyCloudDL4100
ID: YFO3:HRLG:R2RR:FY7F:DYT6:OFHI:HE3U:A2XR:M4D7:Z5SP:YSU7:7534

# docker run -d --name portainer -p 9000:9000 --restart always -v /var/run/docker.sock:/var/run/docker.sock -v /shares/Dockers/portainer:/data portainer/portainer
Unable to find image 'portainer/portainer:latest' locally
latest: Pulling from portainer/portainer
b1c9adb00d32: Pull complete 
8e1ded43c0b4: Pull complete 
1420b03ef5a5: Pull complete 
d21ba0354b51: Pull complete 
10493c971630: Pull complete 
4196812f1636: Pull complete 
Digest: sha256:7037b6f6b22c7c31de621c08f6780b71a3181f5bb2dc0207166625b6b44bae3e
Status: Downloaded newer image for portainer/portainer:latest
beb8dbaa1c0e31dfcdda07dbcf4aa13f159c1f8c918f2067a68a5513645934f2
Error response from daemon: Cannot start container beb8dbaa1c0e31dfcdda07dbcf4aa13f159c1f8c918f2067a68a5513645934f2: [8] System error: invalid argument

[stefaang]

Thanks for the extensive report, this is something I can work on Unfortunately I don't have access to a DLx100 device, but I can try to share a few pointers:

• are you able to create an volume with docker volume create test
• it the daemon running? does the socket exist? ls -l /var/run/docker.sock
• the issetup test should be removed/updated, I'm no longer using a btrfs volume
• docker requires the 3.10 kernel, but it still could be that your kernel lacks a few features... you could try to use a newer one: https://github.com/michaelroland/wdnas-dl2100-kernel

[roblandry]

To follow up, volumes CAN be created and used. However when mapping a volume to a special file it fails. docker.sock timezone, etc. I just tested with this:

docker run -dit --name organizr --restart unless-sto
pped -e "TZ=America/New_York" -p 86:80 -v /shares/Volume_1/Dockers/organizr:/con
fig organizrtools/organizr-v2:latest

and it worked fine.

Docker sock does exist.

ls -la /var/run/docker.sock 
srw-rw----    1 root     root             0 Jan 25 15:50 /var/run/docker.sock

I would love to test a new kernel, but i looking at the link it requires debian stretch, which the dl is not, i dont think .

I am going to dig a little bit about how to get a newer kernel. Like i mentioned before, mounting docker.sock even with the stock docker failed, but the generic error doesnt provide much to go on.

I do however wonder if this may be the root of the issue (from the log above)

time="2020-01-25T10:35:56.144466973-08:00" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"

For testing purposes, i did manually edit /etc/group and added a docker group. Upon restart it is reverted. :-(

[stefaang]

Did you try with the privileged flag? Please be aware of the risk this flag holds. https://docs.docker.com/engine/reference/run/#runtime-privilege-and-linux-capabilities

[NxP4Code]

I tried to run with privileged flag and still got the same error.

root@WDMyCloudDL2100 _docker # docker run -d --name portainer2 -p 9000:9000 --restart always --privileged -v /var/run/docker.sock:/var
/run/docker.sock -v /shares/Dockers/portainer:/data portainer/portainer

4dbf4e05b546620b75d6d51d881a46f6352fb02882726ea0d7af40ac522c791e
docker: Error response from daemon: OCI runtime create failed: container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \\\"/shares/Dockers/portainer\\\" to rootfs \\\"/mnt/HD/HD_a2/Nas_Prog/_docker/vfs/dir/7eb7a4e5f3d284292ccd3dcc93d918307468ceb9c116c871f84db2d593147284\\\" at \\\"/mnt/HD/HD_a2/Nas_Prog/_docker/vfs/dir/7eb7a4e5f3d284292ccd3dcc93d918307468ceb9c116c871f84db2d593147284/data\\\" caused \\\"invalid argument\\\"\"": unknown.
root@WDMyCloudDL2100 _docker #

[roblandry]

Sadly I can no longer test out try anything else as my dl4100 no longer boots. Big ol paper weight. Not sure what happened to it but it's toast.