search

pi-hole / docker-pi-hole

Pi-hole in a docker container
https://pi-hole.net
Other
8.61k stars 1.13k forks source link

Pihole won't start in docker: Failed to set capabilities on file `/usr/bin/pihole-FTL' (Operation not supported) #1085

Open JohnDoeIsKIng opened 2 years ago

JohnDoeIsKIng commented 2 years ago

Since a week I'm trying to get pihole:latest running on a docker/portainer combination on DSM7. I updated from DSM6, portainer 1.24 and a very old pihole (5.*).

Details

I've been trying portainer (marked as deprecated) and portainer-ce on docker. While starting, first I got the error

ERROR: Unable to set capabilities for pihole-FTL. Cannot run as non-root.
If you are seeing this error, please set the environment variable 'DNSMASQ_USER' to the value 'root'

Doing so, now I'm running into

Failed to set capabilities on file `/usr/bin/pihole-FTL' (Operation not supported)
The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file

System info:

Linux DS918plus 4.4.180+ #42661 SMP Fri Apr 1 15:33:06 CST 2022 x86_64 GNU/Linux synology_apollolake_918+

Related Issues

How to reproduce the issue

  1. Environment data

    • Operating System: DSM 7.1-42661 Update 1
    • Hardware: Synology DS918+
    • Kernel Architecture: x86_64
    • Docker Install Info and version:
    • Software source: docker.io
    • Supplimentary Software: Portainer 1.24 and 2
    • Hardware architecture: x86

  2. docker-compose.yml contents, docker run shell command, or paste a screenshot of any UI based configuration of containers here

  4. any additional info to help reproduce

I've tried pihole releases back to 2021.9 - none started without DSNMASQ_USER=root and none did run pihole-FTL. I've tried with portainer and portainer-ce. Before any creation of the container from scratch I deleted my volume binds and recreated them empty.

These common fixes didn't work for my issue

If the above debugging / fixes revealed any new information note it here. Add any other debugging steps you've taken or theories on root cause that may help.

rdwebdesign commented 2 years ago

Please, what is the output of docker version command?

JohnDoeIsKIng commented 2 years ago

Thanks for replying!

Client:
 Version:           20.10.3
 API version:       1.41
 Go version:        go1.17.1
 Git commit:        55f0773
 Built:             Wed Feb  9 04:04:10 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server:
 Engine:
  Version:          20.10.3
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.1
  Git commit:       b487c8f
  Built:            Wed Feb  9 04:04:31 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.4.3
  GitCommit:        3fa00912415f3e9c6f82dd72119179d599efd13b
 runc:
  Version:          v1.0.0-rc93
  GitCommit:        31cc25f16f5eba4d0f53e35374532873744f4b31
 docker-init:
  Version:          0.19.0
  GitCommit:        ed96d00

Hope it helps. Best regards

JD.

JohnDoeIsKIng commented 2 years ago

Update:

I tried something on the console:

root@pihole:/# getcap /usr/bin/pihole-FTL 
Failed to get capabilities of file '/usr/bin/pihole-FTL' (Operation not supported)

I guess if getting file caps isn't possible setting them is neither. An inspection of the container showed caps AUDIT_WRITE, CHOWN, DAC_OVERRIDE, FOWNER, FSETID, KILL, MK added and AUDIT_CONTROL, BLOCK_SUSPEND, DAC_READ_SEARCH, IPC_LOC dropped. Some findings on the internet showed a relation of the error to the NET_BIND_SERVICE cap. Maybe somebody could test this as I'm not able to do so ? Best regards

JD.

rdwebdesign commented 2 years ago

If you are using Portainer, you can set them using "Capabilities" tab, in Portainer web interface.

Pi-hole image should run using this capabilities: PortainerCAPS

Note: You should disable "Privileged mode" option, if it's enabled. PortainerPrivileged

JohnDoeIsKIng commented 2 years ago

Thanx for replying,

I've set the caps exactly as in your hint - no change, pihole-FTL doesn't run. Also I've read before about the privileged mode and so I've never had it turned on. Do you eventually know an image version that 's supposed to run in docker on a synology ? Regards

JD.

rdwebdesign commented 2 years ago

Sorry, I can't answer that. I never used a Synology device.

You could try to ask this at our Discourse Forum.

dschaper commented 2 years ago

Have you tried without Portainer? I'll try to set up a similar configuration on my DS, I need to update to v7 though.

dschaper commented 2 years ago

Can you post your environment settings?

JohnDoeIsKIng commented 2 years ago

Have you tried without Portainer? I'll try to set up a similar configuration on my DS, I need to update to v7 though.

No, I haven't yet. Here 's my environment: pihole-env Best regards

JD.

JohnDoeIsKIng commented 2 years ago

Update 2:

I've searched around further and found this: https://github.com/alexbers/mtprotoproxy/issues/157#issuecomment-547590932 Could that be the cause for my problem ? Because on my system:

Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 2
  Running: 2
  Paused: 0
  Stopped: 0
 Images: 2
 Server Version: 20.10.3
 Storage Driver: aufs
  Root Dir: /volume1/@docker/aufs
  Backing Filesystem: extfs
  Dirs: 16
  Dirperm1 Supported: true
 Logging Driver: db
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs db fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3fa00912415f3e9c6f82dd72119179d599efd13b
 runc version: 31cc25f16f5eba4d0f53e35374532873744f4b31
 init version: ed96d00 (expected: de40ad0)
 Security Options:
  apparmor
 Kernel Version: 4.4.180+
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 3.688GiB
 Name: DS918plus
 ID: GAJ3:THYJ:AK7H:EJHU:NYEF:ZFOS:FBCU:B4VU:NP4R:AWCZ:XYRI:B62F
 Docker Root Dir: /volume1/@docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No kernel memory TCP limit support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
WARNING: No blkio weight support
WARNING: No blkio weight_device support
WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support
WARNING: the aufs storage-driver is deprecated, and will be removed in a future release.

I've tried the manual from the link https://docs.docker.com/storage/storagedriver/overlayfs-driver/, but my docker-fs is still aufs. Eventually somebody with a running pihole on docker could check if the kernel config of the pihole-container has

CONFIG_AUFS_XATTR=y

Regards

JD.

dschaper commented 2 years ago

Yes, if AUFS does not allow capabilities to be set then there will be issues starting.

And we don't configure the kernel in the image, it uses the kernel of the host OS.

JohnDoeIsKIng commented 2 years ago

Okay, that means for me pihole on docker/synology isn't an option any more. As a last try I've edited

/var/packages/Docker/etc/dockerd.json

hoping to change the storage driver from aufs to overlay2 or btrfs but nothing changed - driver is still aufs. So I'll remove pihole from docker and probably install it on another hardware. Sad but true. May I'll try to set it up without portainer as in https://discourse.pi-hole.net/t/setup-on-synology-docker/18067. Regards

JD.

dschaper commented 2 years ago

There are a number of users that have Pi-hole running on Synology. Do you know what filesystem you have the Synology set to use?

My Synology Docker setup shows 'btrfs' since I'm using the HybridRaid format with my 2 disk unit. It sounds like you may have ext4 format which ends up as 'aufs' under Docker.

JohnDoeIsKIng commented 2 years ago

You've supposed right - All of the hdds have ext4-fs. I'm wondering where the issue comes from since before the update to DSM7.1-42661 there weren't any issues with cap settings. I guess if I have to change one hdd to btrfs that I have to format that one and remove temporarily the data on it?

rdwebdesign commented 2 years ago

... I'm wondering where the issue comes from since before the update to DSM7.1-42661 there weren't any issues with cap settings.

I'm not sure, but this could be a combination of different things.

Recently (before v20.10.14), docker discovered a bug (https://github.com/moby/moby/security/advisories/GHSA-2mm7-x5h6-5pvq) and changed the way capabilities are inherited. To address this change, we changed how pi-hole images handle capabilities. We also changed from buster base to bullseye base image.

Any of these changes (or any combination of them) could be the reason why you are seeing different behavior after the upgrade.

JohnDoeIsKIng commented 2 years ago

Dear community,

I've managed to get pihole:latest running on docker/portainer. I've transformed the file system of one hdd into btrfs and it worked nearly out of the box. So bringing this to an end, I've one last question: I have a copy of my pihole-folder of the old version I used to run. Is it possible to import at least the block lists and my edited white and black lists into the new version ? I have NO teleporter tar file, just the content of the folder:

d---------+ 3 0 0       4096 May  1 17:21 .
d---------+ 4 0 0       4096 May  1 17:26 ..
----------+ 1 0 0        242 May  1 17:21 adlists.list
----------+ 1 0 0        596 May  1 17:21 dns-servers.conf
----------+ 1 0 0         17 May  1 17:21 GitHubVersions
----------+ 1 0 0  139624448 May  1 17:21 gravity.db
----------+ 1 0 0    2501728 May  1 17:21 list.0.raw.githubusercontent.com.domains
----------+ 1 0 0     594616 May  1 17:21 list.1.mirror1.malwaredomains.com.domains
----------+ 1 0 0        521 May  1 17:21 list.2.s3.amazonaws.com.domains
----------+ 1 0 0      43529 May  1 17:21 list.3.s3.amazonaws.com.domains
----------+ 1 0 0   45056207 May  1 17:21 list.4.raw.githubusercontent.com.domains
----------+ 1 0 0         20 May  1 17:21 localbranches
----------+ 1 0 0         31 May  1 17:21 local.list
----------+ 1 0 0         39 May  1 17:21 localversions
d---------+ 2 0 0       4096 May  1 17:21 migration_backup
----------+ 1 0 0          0 May  1 17:21 pihole-FTL.conf
----------+ 1 0 0 1214263296 May  1 17:21 pihole-FTL.db
----------+ 1 0 0        443 May  1 17:21 setupVars.conf
----------+ 1 0 0        443 May  1 17:21 setupVars.conf.update.bak

Do I have to re-initialize some databases ? Best regards

JD.

sancoder commented 2 years ago

Why not make some extra code to detect Synology just for this bug? Is it too much a deviation?

I was trying to upgrade from pihole installed back in 2019 to something fresher, and bumped into this change of behavior. Before upgrade - pihole was running successfully in docker on Synology, and after upgrading it cannot start because pihole cannot set capabilities on file '/usr/bin/pihole-FTL' just as in the first message of this thread. Yes, I have ext4 FS, and I don't have an option to reformat all of HDDs in NAS.

Or, introduce an env variable - something like $DONT_FIX_CAPS defaulting to 0 and if a user sets it as 1, then code in fix_capabilities just ignores error coming from setcap call.

dschaper commented 2 years ago

pihole-FTL won't run without the capabilities.

sancoder commented 2 years ago

Error from setcap doesn't mean pihole-FTL not capable to do what asked.

PromoFaux commented 2 years ago

@sancoder - There were some changes made in https://github.com/pi-hole/docker-pi-hole/pull/1086 that might (strong might) fix the issues you're seeing

Have just tagged 2022.05, which is just building (though if you're super desperate to try it before it builds, you can try the :dev tag

sancoder commented 2 years ago

@PromoFaux thanks, still not working as non-root (adding DNSMASQ_USER=root works).

pihole2022 | [s6-init] making user provided files available at /var/run/s6/etc...exited 0.
pihole2022 | [s6-init] ensuring user provided files have correct perms...exited 0.
pihole2022 | [fix-attrs.d] applying ownership & permissions fixes...
pihole2022 | [fix-attrs.d] 01-resolver-resolv: applying...
pihole2022 | [fix-attrs.d] 01-resolver-resolv: exited 0.
pihole2022 | [fix-attrs.d] done.
pihole2022 | [cont-init.d] executing container initialization scripts...
pihole2022 | [cont-init.d] 05-changer-uid-gid.sh: executing...
pihole2022 | [cont-init.d] 05-changer-uid-gid.sh: exited 0.
pihole2022 | [cont-init.d] 20-start.sh: executing...
pihole2022 |  ::: Starting docker specific checks & setup for docker pihole/pihole
pihole2022 | cap[cap_net_admin] not permitted
pihole2022 | cap[cap_sys_nice] not permitted
pihole2022 | Failed to set capabilities on file `/usr/bin/pihole-FTL' (Operation not supported)
pihole2022 | The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file
pihole2022 | ERROR: Unable to set capabilities for pihole-FTL. Cannot run as non-root.
pihole2022 |        If you are seeing this error, please set the environment variable 'DNSMASQ_USER' to the value 'root'
pihole2022 | [cont-init.d] 20-start.sh: exited 1.
pihole2022 | [cont-finish.d] executing container finish scripts...
pihole2022 | [cont-finish.d] done.
pihole2022 | [s6-finish] waiting for services.
pihole2022 | [s6-finish] sending all processes the TERM signal.
pihole2022 | [s6-finish] sending all processes the KILL signal and exiting.

For others who found this issue because of the error message: if docker is on Synology, please see #963

b-m-f commented 2 years ago

I am having the same issue on a raspberry pi, running pihole with Podman.

From all the approaches I have found only setting the ENV variable DNSMASQ_USER=root seems to do the trick for now.

dschaper commented 2 years ago

I am having the same issue on a raspberry pi, running pihole with Podman.

What versions of the OS and Podman?

b-m-f commented 2 years ago