torrent mmap_span.copyBytes panic on low disk space

[battlmonstr]

System information

Erigon version: ./erigon --version

devel

OS & Version: Linux

Commit hash: ab04de6d9f9fc4291dbfd113dadc79e798fd8daf

Erigon Command (with flags/config):

--prune htrc
--snapshots=false
--internalcl

Chain/Network: mainnet

Expected behaviour

Produce an error in the log. It could be a fatal error, but it's better to degrade the functionality.

Actual behaviour

Crash.

Steps to reproduce the behaviour

Run with datadir having 700 Gb limit or less.

Backtrace

goroutine 18622 [running]:
runtime.throw({0x25ab165?, 0x108edd39a6ef5956?})
    runtime/panic.go:1077 +0x5c fp=0xc050adaa00 sp=0xc050ada9d0 pc=0x45b11c
runtime.sigpanic()
    runtime/signal_unix.go:858 +0x116 fp=0xc050adaa60 sp=0xc050adaa00 pc=0x472136
runtime.memmove()
    runtime/memmove_amd64.s:375 +0x43c fp=0xc050adaa68 sp=0xc050adaa60 pc=0x49295c
github.com/anacrolix/torrent/mmap_span.copyBytes(...)
    github.com/anacrolix/torrent@v1.52.6-0.20230926121951-11833b45cfbe/mmap_span/mmap_span.go:79

see full log attachment:

erigon.log.zip

[AskAlexSharov]

FYI: https://github.com/ledgerwatch/erigon/pull/10074

[battlmonstr]

@AskAlexSharov could you explain how https://github.com/ledgerwatch/erigon/pull/10074 is related to this, and why the code there is commented out?

[AskAlexSharov]

@battlmonstr because i decided to stay on MMAP https://github.com/ledgerwatch/erigon/pull/10074#issuecomment-2080359603

[anacrolix]

If it helps it might be possible to prevent this from panicking by passing a flag to mmap causing the space to be preallocated, rather than paging it in on demand. This would cause the error to occur when the torrent opens the storage, which is when the info bytes are obtained and added. So you you should see an error when you call Client.AddTorrent or something instead.

[AskAlexSharov]

Can be:

• we are anyway unfriendly for CoW FileSystems
• and pre-allocation may reduce fragmentation of file on disk

[anacrolix]

Oh yeah, CoW FS might still panic on this. I don't think it's viable to catch a signal writing to a mmap in Go.