BUG: Disk usage in docker / MacOS is incorrect

[ErebusBat]

I have an agent running on macOS 15.1 / docker (orbstack). I have a USB drive mounted in the extra-filesystems.

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /Volumes/SeaBack:/extra-filesystems/SeaBack:ro

Beszel (0.7.2 agent and hub) is reporting a VASTLY incorrect disk usage:

[henrygd]

Yeah, I'd say so. Unfortunately I don't have a Mac device to troubleshoot myself.

Please restart the agent with LOG_LEVEL=debug and let me know what it outputs for partitions and diskstats.

From a quick google it looks like this may be an issue with how Docker works on MacOS: https://forums.docker.com/t/macos-docker-bind-mount-reports-incorrect-free-space/137982/4

Can you test with the binary agent to see if it has the same problem? I think you would add EXTRA_FILESYSTEMS=/Volumes/SeaBack or try the actual device / partition, like EXTRA_FILESYSTEMS=disk4s1.

[ErebusBat]

Seeing that you are using Docker, I would like to recommend Servbay to you, which is more professional and easy to use than Docker on Mac.

Thank you for the recommendation; however I am using OrbStack after evaluating several options for my day job we found that OrbStack was the best cost/$.

[ErebusBat]

@henrygd thank you for your response. Please find my troubleshooting steps below:

Env Var Adding the EXTRA_FILESYSTEMS env var did not seem to have any effect, positive or negative. I assume because it is already mounted under /extra-filesystems but I do not know that for sure.

Debug Logging Raw logs:

beszel-agent  | 2024/11/04 16:09:14 DEBUG 0.7.2
beszel-agent  | 2024/11/04 16:09:14 DEBUG Not monitoring ZFS ARC err="open /proc/spl/kstat/zfs/arcstats: no such file or directory"
beszel-agent  | 2024/11/04 16:09:14 DEBUG Disk partitions="[{\"device\":\"/dev/vdb1\",\"mountpoint\":\"/etc/resolv.conf\",\"fstype\":\"btrfs\",\"opts\":[\"rw\",\"noatime\",\"bind\"]} {\"device\":\"/dev/vdb1\",\"mountpoint\":\"/etc/hostname\",\"fstype\"
:\"btrfs\",\"opts\":[\"rw\",\"noatime\",\"bind\"]} {\"device\":\"/dev/vdb1\",\"mountpoint\":\"/etc/hosts\",\"fstype\":\"btrfs\",\"opts\":[\"rw\",\"noatime\",\"bind\"]}]"
beszel-agent  | 2024/11/04 16:09:14 DEBUG Disk I/O diskstats="map[vda:{\"readCount\":20215,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":722542592,\"writeBytes\":0,\"readTime\":591,\"writeTime\":0,\"iopsInProgress\":0,\"io
Time\":3465,\"weightedIO\":591,\"name\":\"vda\",\"serialNumber\":\"\",\"label\":\"\"} vdb:{\"readCount\":114217,\"mergedReadCount\":2322,\"writeCount\":4506,\"mergedWriteCount\":3890,\"readBytes\":7143862272,\"writeBytes\":125472768,\"readTime\":961,\"
writeTime\":5907,\"iopsInProgress\":0,\"ioTime\":19790,\"weightedIO\":15522,\"name\":\"vdb\",\"serialNumber\":\"\",\"label\":\"\"} vdb1:{\"readCount\":114207,\"mergedReadCount\":2322,\"writeCount\":4506,\"mergedWriteCount\":3890,\"readBytes\":714382131
2,\"writeBytes\":125472768,\"readTime\":961,\"writeTime\":5907,\"iopsInProgress\":0,\"ioTime\":20982,\"weightedIO\":9672,\"name\":\"vdb1\",\"serialNumber\":\"\",\"label\":\"\"} vdc:{\"readCount\":3,\"mergedReadCount\":0,\"writeCount\":1,\"mergedWriteCo
unt\":0,\"readBytes\":12288,\"writeBytes\":4096,\"readTime\":0,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":0,\"weightedIO\":0,\"name\":\"vdc\",\"serialNumber\":\"\",\"label\":\"\"} zram0:{\"readCount\":2,\"mergedReadCount\":0,\"writeCount\":1,\"mer
gedWriteCount\":0,\"readBytes\":8192,\"writeBytes\":4096,\"readTime\":0,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":0,\"weightedIO\":0,\"name\":\"zram0\",\"serialNumber\":\"\",\"label\":\"\"}]"
beszel-agent  | 2024/11/04 16:09:14 INFO Detected root device name=vdb1
beszel-agent  | 2024/11/04 16:09:14 DEBUG /extra-filesystems mountpoint=/extra-filesystems/SeaBack
beszel-agent  | 2024/11/04 16:09:14 WARN Device not found in diskstats name=SeaBack
beszel-agent  | 2024/11/04 16:09:14 INFO Detected network interface name=eth0 sent=2695 recv=938857
beszel-agent  | 2024/11/04 16:09:14 INFO Detected network interface name=.orbmirror0 sent=746 recv=242169
beszel-agent  | 2024/11/04 16:09:14 INFO Detected network interface name=.orbmirror1 sent=746 recv=242169
beszel-agent  | 2024/11/04 16:09:14 DEBUG Docker version=27.3.1 concurrency=5
beszel-agent  | 2024/11/04 16:09:14 DEBUG Getting stats
beszel-agent  | 2024/11/04 16:09:14 DEBUG Temperature sensors=[]
beszel-agent  | 2024/11/04 16:09:14 DEBUG sysinfo data="{Hostname:orbstack KernelVersion:6.11.5-orbstack-00280-g96d99c92a42b Cores:1 Threads:1 CpuModel: Uptime:9308 Cpu:100 MemPct:33.21 DiskPct:2.65 Bandwidth:0 AgentVersion:0.7.2}"
beszel-agent  | 2024/11/04 16:09:14 DEBUG System stats data="{Stats:{Cpu:100 MaxCpu:0 Mem:3.89 MemUsed:1.29 MemPct:33.21 MemBuffCache:0.3 MemZfsArc:0 Swap:4.89 SwapUsed:0 DiskTotal:244.69 DiskUsed:6.49 DiskPct:2.65 DiskReadPs:0 DiskWritePs:1.85 MaxDisk
ReadPs:0 MaxDiskWritePs:0 NetworkSent:0 NetworkRecv:0 MaxNetworkSent:0 MaxNetworkRecv:0 Temperatures:map[] ExtraFs:map[]} Info:{Hostname:orbstack KernelVersion:6.11.5-orbstack-00280-g96d99c92a42b Cores:1 Threads:1 CpuModel: Uptime:9308 Cpu:100 MemPct:3
3.21 DiskPct:2.65 Bandwidth:0 AgentVersion:0.7.2} Containers:[]}"
beszel-agent  | 2024/11/04 16:09:14 DEBUG Docker stats data="[0x400014c8c0 0x400014c9a0 0x400014ca80 0x400014cb60 0x400014c2a0 0x400014c5b0 0x400014c7e0]"
beszel-agent  | 2024/11/04 16:09:14 DEBUG Extra filesystems data=map[SeaBack:0x40000a90a0]
beszel-agent  | 2024/11/04 16:09:14 DEBUG Stats data="{Stats:{Cpu:100 MaxCpu:0 Mem:3.89 MemUsed:1.29 MemPct:33.21 MemBuffCache:0.3 MemZfsArc:0 Swap:4.89 SwapUsed:0 DiskTotal:244.69 DiskUsed:6.49 DiskPct:2.65 DiskReadPs:0 DiskWritePs:1.85 MaxDiskReadPs:
0 MaxDiskWritePs:0 NetworkSent:0 NetworkRecv:0 MaxNetworkSent:0 MaxNetworkRecv:0 Temperatures:map[] ExtraFs:map[SeaBack:0x40000a90a0]} Info:{Hostname:orbstack KernelVersion:6.11.5-orbstack-00280-g96d99c92a42b Cores:1 Threads:1 CpuModel: Uptime:9308 Cpu
:100 MemPct:33.21 DiskPct:2.65 Bandwidth:0 AgentVersion:0.7.2} Containers:[0x400014c8c0 0x400014c9a0 0x400014ca80 0x400014cb60 0x400014c2a0 0x400014c5b0 0x400014c7e0]}"
beszel-agent  | 2024/11/04 16:09:14 INFO Starting SSH server address=:45876
beszel-agent  | 2024/11/04 16:09:46 DEBUG Getting stats
beszel-agent  | 2024/11/04 16:09:46 DEBUG Temperature sensors=[]
beszel-agent  | 2024/11/04 16:09:46 DEBUG sysinfo data="{Hostname:orbstack KernelVersion:6.11.5-orbstack-00280-g96d99c92a42b Cores:1 Threads:1 CpuModel: Uptime:9339 Cpu:6.39 MemPct:33.22 DiskPct:2.65 Bandwidth:0 AgentVersion:0.7.2}"
beszel-agent  | 2024/11/04 16:09:46 DEBUG System stats data="{Stats:{Cpu:6.39 MaxCpu:0 Mem:3.89 MemUsed:1.29 MemPct:33.22 MemBuffCache:0.28 MemZfsArc:0 Swap:4.89 SwapUsed:0 DiskTotal:244.69 DiskUsed:6.49 DiskPct:2.65 DiskReadPs:0.13 DiskWritePs:0.02 Ma
xDiskReadPs:0 MaxDiskWritePs:0 NetworkSent:0 NetworkRecv:0 MaxNetworkSent:0 MaxNetworkRecv:0 Temperatures:map[] ExtraFs:map[]} Info:{Hostname:orbstack KernelVersion:6.11.5-orbstack-00280-g96d99c92a42b Cores:1 Threads:1 CpuModel: Uptime:9339 Cpu:6.39 Me
mPct:33.22 DiskPct:2.65 Bandwidth:0 AgentVersion:0.7.2} Containers:[]}"
beszel-agent  | 2024/11/04 16:09:46 DEBUG Docker stats data="[0x400014cb60 0x400014c2a0 0x400014c5b0 0x400014c7e0 0x400014c8c0 0x400014de30 0x400014ca80]"
beszel-agent  | 2024/11/04 16:09:46 DEBUG Extra filesystems data=map[SeaBack:0x40000a90a0]

Here is a prettied / decoded disk partitions:

[
  {
    "device": "/dev/vdb1",
    "mountpoint": "/etc/resolv.conf",
    "fstype": "btrfs",
    "opts": [
      "rw",
      "noatime",
      "bind"
    ]
  },
  {
    "device": "/dev/vdb1",
    "mountpoint": "/etc/hostname",
    "fstype": "btrfs",
    "opts": [
      "rw",
      "noatime",
      "bind"
    ]
  },
  {
    "device": "/dev/vdb1",
    "mountpoint": "/etc/hosts",
    "fstype": "btrfs",
    "opts": [
      "rw",
      "noatime",
      "bind"
    ]
  }
]

And Diskstats:

vda:
{\"readCount\":20215,\"mergedReadCount\":0,\"writeCount\":0,\"mergedWriteCount\":0,\"readBytes\":722542592,\"writeBytes\":0,\"readTime\":591,\"writeTime\":0,\"iopsInProgress\":0,\"io Time\":3465,\"weightedIO\":591,\"name\":\"vda\",\"serialNumber\":\"\",\"label\":\"\"}

vdb:
{\"readCount\":114217,\"mergedReadCount\":2322,\"writeCount\":4506,\"mergedWriteCount\":3890,\"readBytes\":7143862272,\"writeBytes\":125472768,\"readTime\":961,\" writeTime\":5907,\"iopsInProgress\":0,\"ioTime\":19790,\"weightedIO\":15522,\"name\":\"vdb\",\"serialNumber\":\"\",\"label\":\"\"}

vdb1:
{\"readCount\":114207,\"mergedReadCount\":2322,\"writeCount\":4506,\"mergedWriteCount\":3890,\"readBytes\":7143821312,\"writeBytes\":125472768,\"readTime\":961,\"writeTime\":5907,\"iopsInProgress\":0,\"ioTime\":20982,\"weightedIO\":9672,\"name\":\"vdb1\",\"serialNumber\":\"\",\"label\":\"\"}

vdc:
{\"readCount\":3,\"mergedReadCount\":0,\"writeCount\":1,\"mergedWriteCount\":0,\"readBytes\":12288,\"writeBytes\":4096,\"readTime\":0,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":0,\"weightedIO\":0,\"name\":\"vdc\",\"serialNumber\":\"\",\"label\":\"\"}

zram0:
{\"readCount\":2,\"mergedReadCount\":0,\"writeCount\":1,\"mergedWriteCount\":0,\"readBytes\":8192,\"writeBytes\":4096,\"readTime\":0,\"writeTime\":0,\"iopsInProgress\":0,\"ioTime\":0,\"weightedIO\":0,\"name\":\"zram0\",\"serialNumber\":\"\",\"label\":\"\"}]"

Binary Agent As was probably expected things seem to work correctly under docker... including temperature sensors which I had not realized were not reading under docker (but is obvious in retrospect).

[henrygd]

Thanks!

It looks like the dockerized agent can't see the underlying device for /Volumes/SeaBack. The only partition it sees is /dev/vdb1, which I assume is the default storage device for the container.

Can you paste your output of lsblk here? Or whatever the Mac equivalent is. Maybe diskutil list and df?

Temperature sensors populate in Docker on Linux, so I think that's another issue that's specific to Docker on MacOS.

EXTRA_FILESYSTEMS should only be used for the binary agent. If you specify EXTRA_FILESYSTEMS=vda,vdc that should add those devices as extra disks.