docker-compose monerod File too large

[korneliuszo]

After some syncing I get:

W Failed to commit a transaction to the db: File too large E Exception in cleanup_handle_incoming_blocks: Failed to commit a transaction to the db: File too large

db on disk is using 17GB monerod process uses 32GB virtual memory when error occures

OS: manjaro aarch64 docker -v Docker version 23.0.3, build 3e7cbfdee1 docker-compose version Docker Compose version 2.17.2

[SChernykh]

You don't have enough disc space for your docker container.

[korneliuszo]

df -h says I have >800GB free on /.

[SChernykh]

Googling "EFBIG" file too large suggests that you need to adjust ulimit for maximum file size. Run ulimit -f to check your current file size limit (in kilobytes).

[korneliuszo]

ulimit -f inside container says unlimited

[SChernykh]

And what about outside container? It's an OS limitation in your setup, and this is not directly a P2Pool issue.

What's the current size of data.mdb in the container? I see you wrote 17 GB in the first post, so it's probably an OS limitation which also works in the container.

[bladedoyle]

@korneliuszo Can you please check which partition the docker volume is on, and double check that it has free space? First run docker volume inspect monero | grep Mountpoint then run sudo du -h /the/directory/it/shows

If you have enough space the next thing to look at is filesystem type by running something like sudo stat --format=%T --file-system /the/directory/it/shows to make sure its not ntfs or FAT or something.

[korneliuszo]

sudo du -h ... says 14G used stat says ext2/ext3 I have one root partition with 887G free space

[bladedoyle]

Hm, I had wanted sudo df -h /the/directory/it/shows but somehow typed "du".

[korneliuszo]

it shows root mountpoint with >800G free space

[bladedoyle]

are you running the docker daemon as root (this is the default)? ps -ef | grep dockerd what storage driver is docker using? docker info | grep -A 4 "Storage Driver" "manjaro aarch64" - is this an ARM architecture?

[korneliuszo]

docker run from systemd, runs as root

docker info | grep -A 4 "Storage Driver"
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true

it is arm64

[bladedoyle]

Thanks for all the info. I dont see anything that would cause this issue. Maybe it is some bug or configuration in arm64 docker or monerod. I suggest testing docker volumes first, to see if the limitation is imposed by docker or monerod:

docker run -it --rm -v monerod:/mnt monero:latest dd if=/dev/zero of=/mnt/foo bs=1G count=32 status=progress

and then, to clean up the test file created by that command:

docker run -it --rm -v monerod:/mnt monero:latest rm -f /mnt/foo

[korneliuszo]

docker run -it --rm -v monerod:/mnt --entrypoint bash monero:latest monero@ff2a440ad0aa:~$ dd if=/dev/zero of=/mnt/foo bs=1G count=32 status=progress 34359738368 bytes (34 GB, 32 GiB) copied, 93 s, 368 MB/s 32+0 records in 32+0 records out 34359738368 bytes (34 GB, 32 GiB) copied, 93.3605 s, 368 MB/s monero@ff2a440ad0aa:~$ rm -f /mnt/foo

[bladedoyle]

This could be related? https://github.com/monero-project/monero/issues/8473#issuecomment-1220612614

[SChernykh]

So it's an unstable hardware in the end?