search

pi-hole / docker-pi-hole

Pi-hole in a docker container
https://pi-hole.net
Other
8.64k stars 1.14k forks source link

[macOS] Disk shortage bug #951

Open VicSegers opened 2 years ago

VicSegers commented 2 years ago

This is a: Bug

Details

When updating to "Pi-hole FTL v5.12, Web v5.9 and Core v5.7" I received this message after running:

pi-hole disk shortage bug

Related Issues

How to reproduce the issue

  1. Environment data

    • Operating System: macOS 12.1
    • Hardware: MacBook Pro (2016)
    • Kernel Architecture: Darwin 21.2.0
    • Docker Install Info and version:
    • Software source: Homebrew
    • Supplimentary Software: /
    • Hardware architecture: x86_64

  2. docker-compose.yml contents, docker run shell command, or paste a screenshot of any UI based configuration of containers here

docker-compose.yml

version: "3"

# More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/
services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "67:67/udp"
      - "80:80/tcp"
      - "443:443/tcp"
    environment:
      TZ: 'Belgium/Brussels'
      WEBPASSWORD: 'foo'
    # Volumes store your data between container upgrades
    volumes:
      - './etc-pihole/:/etc/pihole/'
      - './etc-dnsmasq.d/:/etc/dnsmasq.d/'
    # Recommended but not required (DHCP needs NET_ADMIN)
    #   https://github.com/pi-hole/docker-pi-hole#note-on-capabilities
    cap_add:
      - NET_ADMIN
    restart: unless-stopped
  1. any additional info to help reproduce

These common fixes didn't work for my issue

If the above debugging / fixes revealed any new information note it here. Add any other debugging steps you've taken or theories on root cause that may help.

PromoFaux commented 2 years ago

What does running df -h from inside the container output?

@DL6ER could there be something funky in FTLs detection here?

DL6ER commented 2 years ago

Also, what is the output of

stat -f /etc/pihole

inside the container?

FTL computes the size of the directory by computing the fragment size times the block size: https://github.com/pi-hole/FTL/blob/83a950b59062b5cf5f90b58abbf6842b6624ca65/src/files.c#L168

VicSegers commented 2 years ago

What does running df -h from inside the container output?

@DL6ER could there be something funky in FTLs detection here?

df -h in the container:

Filesystem      Size  Used Avail Use% Mounted on
overlay          59G  3.8G   52G   7% /
tmpfs            64M     0   64M   0% /dev
tmpfs           994M     0  994M   0% /sys/fs/cgroup
shm              64M  1.2M   63M   2% /dev/shm
grpcfuse        466G  213G  254G  46% /etc/pihole
/dev/vda1        59G  3.8G   52G   7% /etc/hosts
tmpfs           994M     0  994M   0% /proc/acpi
tmpfs           994M     0  994M   0% /sys/firmware

and du -h on my host

8.0K    ./etc-dnsmasq.d
4.0K    ./etc-pihole/migration_backup
647M    ./etc-pihole
647M    .
VicSegers commented 2 years ago

Also, what is the output of

stat -f /etc/pihole

inside the container?

FTL computes the size of the directory by computing the fragment size times the block size: https://github.com/pi-hole/FTL/blob/83a950b59062b5cf5f90b58abbf6842b6624ca65/src/files.c#L168

  File: "/etc/pihole/"
    ID: 0        Namelen: 255     Type: fuseblk
Block size: 1048576    Fundamental block size: 4096
Blocks: Total: 122061322  Free: 66413177   Available: 66413177
Inodes: Total: 2659341079 Free: 2656527080
DL6ER commented 2 years ago

grpcfuse seems to be some specialty of docker. I suspect it does not report everything properly. I'm already on my way to bed (approaching midnight over here) and will report back with idea soon.

VicSegers commented 2 years ago

Good night! This isn't urgent at all, everything works fine.

PromoFaux commented 2 years ago

It could be a Docker Desktop on Mac thing? Just Copy pasted your compose file and span it up on Docker Desktop for windows:

image

And no such messages:

image

My main install running on a Pi is also not seeing this message.

What version of docker is on the host?

VicSegers commented 2 years ago

Docker version 20.10.10, build b485636

mrcemento commented 2 years ago

Same issue here. Me too on docket on Mac OS. Seems definitely docker on Mac OS related.

Tynox commented 2 years ago

I got the some warning too after updated to the latest version on macOS 12.1. Docker engine: 20.10.11

torrelasley commented 2 years ago

Also getting this message on macOS docker image. This is the first google result for "Disk shortage" related to pihole.

DL6ER commented 2 years ago

It very much looks like docker for macOS bug. Sorry, I looked everywhere but I don't see an obvious solution for this. Even more because neither does anyone from the core developers team has a Mac available for testing nor does it seem easy to detect the misbehaving docker version (well, except for "this value is unrealistically large").

For now, the best I can offer you is setting

CHECK_DISK=0

in /etc/pihole/pihole-FTL.conf (see here). This will disable the test altogether.

I'll come back to you when I have something to test. Have a great time and merry Christmas everyone! :christmas_tree:

PromoFaux commented 2 years ago

To set that variable easily, you can set it as an environment variable e.g FTLCONF_CHECK_DISK=0 in your docker-compose / docker run scripts https://github.com/pi-hole/docker-pi-hole#advanced-variables

VersionThis commented 2 years ago

Just to add a data point that I am seeing this same behavior, running on a fresh install:


Macbook Retina
macOS 12.1 
Darwin Kernel Version 21.1.0
Docker version 20.10.6, build 370c289

root@a3ef6990b77b:~# date ; uname -a ; uptime ; pwd
Sat Dec 25 08:45:47 CST 2021
Linux a3ef6990b77b 5.10.25-linuxkit #1 SMP Tue Mar 23 09:27:39 UTC 2021 x86_64 GNU/Linux
 08:45:47 up 21:02,  0 users,  load average: 0.12, 0.24, 0.39
/root
root@a3ef6990b77b:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay          59G  3.9G   52G   7% /
tmpfs            64M     0   64M   0% /dev
tmpfs           994M     0  994M   0% /sys/fs/cgroup
shm              64M  712K   64M   2% /dev/shm
grpcfuse        466G  277G  189G  60% /etc/pihole
/dev/vda1        59G  3.9G   52G   7% /etc/hosts
tmpfs           994M     0  994M   0% /proc/acpi
tmpfs           994M     0  994M   0% /sys/firmware
root@a3ef6990b77b:~# stat -f /etc/pihole
  File: "/etc/pihole"
    ID: 0        Namelen: 255     Type: fuseblk
Block size: 1048576    Fundamental block size: 4096
Blocks: Total: 122061322  Free: 49524361   Available: 49524361
Inodes: Total: 1983962276 Free: 1980974440
root@a3ef6990b77b:~#

Will disable the check as suggested to suppress this for now.

On a side note, I sooo wish Docker on Mac & Windows was more "production ready" than it is. No wonder the big Clouds have their own container engines now.

Thanks & Happy Holidays. 🎄

pralor-bot commented 2 years ago

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/adminlte-version-is-v5-9-php-socket-0-connection-refused/51839/11

rvanbaalen commented 2 years ago

Not sure if it add something to the discussion but I'm seeing the exact same message. Latest pihole docker container (downloaded today), running Docker on Mac.

For now I've added 

    environment:
      FTLCONF_CHECK_DISK: 0

to the docker-compose.yml file as suggested by @PromoFaux, that'll at least get rid of the message for now.

ghost commented 2 years ago

I too can confirm this has only happened to me on mac docker installs.

github-actions[bot] commented 2 years ago

This issue is stale because it has been open 30 days with no activity. Please comment or update this issue or it will be closed in 5 days.

sellersj commented 2 years ago

Still happening with latest pi-hole 5.9

rdwebdesign commented 2 years ago

What do you see running df -h inside the container?

sellersj commented 2 years ago 
# df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay          59G  2.2G   54G   4% /
tmpfs            64M     0   64M   0% /dev
shm              64M  716K   64M   2% /dev/shm
grpcfuse        466G  406G   60G  88% /etc/pihole
/dev/vda1        59G  2.2G   54G   4% /etc/hosts
tmpfs           2.0G     0  2.0G   0% /proc/acpi
tmpfs           2.0G     0  2.0G   0% /sys/firmware

I also found another command in pi-hole and here's the command and output

# df -B1 / 2> /dev/null | awk 'END{ print $3,$2,$5 }'
2362048512 62725623808 4%

I also tried to run

mapfile -t file_system < <(df -h)

but the space between the < my system didn't like and I'm not sure if this is a bug or just not possible to run like this on the command line. https://github.com/pi-hole/pi-hole/blob/6ffa2ba1b2a68c7b0919689e137017ae344aed3c/advanced/Scripts/piholeDebug.sh#L598

The current warning I see is:

Disk shortage (/etc/pihole/pihole-FTL.db) ahead: 36893004% used /etc/pihole: 18.4EB used, 500.0GB total

rdwebdesign commented 2 years ago

grpcfuse was already discussed on this page, starting here (https://github.com/pi-hole/docker-pi-hole/issues/951#issuecomment-999916277).

For now, as a workaround, you can disable the check using FTLCONF_CHECK_DISK=0 in your docker-compose file or run command.

blumarten commented 2 years ago

was there ever a solution for this? I have a primary pi-hole on a headless mac mini and a secondary on a pi zero. I get the message pretty much every day on the mac and a couple of times per week on the pi

PromoFaux commented 2 years ago

@blumarten Just a few comments up: https://github.com/pi-hole/docker-pi-hole/issues/951#issuecomment-1000866780

blumarten commented 2 years ago

@blumarten Just a few comments up: #951 (comment)

thank you mate

russalex83 commented 2 years ago

[ ]

vi /etc/pihole/pihole-FTL.conf

add variable:

CHECK_DISK=0

https://docs.pi-hole.net/ftldns/configfile/#check_disk

pralor-bot commented 1 year ago

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/false-disk-shortage/60134/2

pralor-bot commented 1 year ago

This issue has been mentioned on Pi-hole Userspace. There might be relevant details there:

https://discourse.pi-hole.net/t/new-install-disk-used-available-by-3million/62113/2

bluehz commented 1 year ago

Still seeing this problem with fresh install. Any resolution?

DL6ER commented 1 year ago

Sorry, we cannot detect the environment outside the container and macOS+docker is apparently still reporting incorrect information here. I'll implement a workaround that will silence the warning in case the results found are not useful, that is used_space >> total_space but be aware that this will only work when total_space is correct. We've seen Pi-holes being installed on large disk array machines with truly available space in the low EB range so we should not define an arbitrary/artificial limit here.