[v2.1.1] `Fatal process out of memory: Oilpan: CagedHeap reservation.` with an address space (`ulimit -v`/`prlimit -v=`) below 64 GB

[benz0li]

Version: Deno 2.1.1

Steps to reproduce

On a host with docker installed

docker run --rm -ti denoland/deno:2.1.1 bash -c 'ulimit -v 67108864; deno'

#
# Fatal process out of memory: Oilpan: CagedHeap reservation.
#
==== C stack trace ===============================

    deno(+0x2d74233) [0x55c12b372233]
    deno(+0x2d73afb) [0x55c12b371afb]
    deno(+0x2d6f008) [0x55c12b36d008]
    deno(+0x2dc4f6b) [0x55c12b3c2f6b]
    deno(+0x2f4543e) [0x55c12b54343e]
    deno(+0x37b2089) [0x55c12bdb0089]
    deno(+0x37bab40) [0x55c12bdb8b40]
    deno(+0x37ba8bf) [0x55c12bdb88bf]
    deno(+0x37b2131) [0x55c12bdb0131]
    deno(+0x670dd6e) [0x55c12ed0bd6e]
    deno(+0x440f6c3) [0x55c12ca0d6c3]
    deno(+0x67a160b) [0x55c12ed9f60b]
    deno(+0x444c02e) [0x55c12ca4a02e]
    deno(+0x469768c) [0x55c12cc9568c]
    deno(+0x4a50309) [0x55c12d04e309]
    deno(+0x453bed9) [0x55c12cb39ed9]
    deno(+0x4534ee7) [0x55c12cb32ee7]
    deno(+0x43f0c90) [0x55c12c9eec90]
    deno(+0x4ad1965) [0x55c12d0cf965]
    /lib/x86_64-linux-gnu/libc.so.6(+0x2724a) [0x7ffad151124a]
    /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x7ffad1511305]
    deno(+0x2d47029) [0x55c12b345029]

Cross reference:

• https://github.com/quarto-dev/quarto-cli/issues/4197#issuecomment-2504557128 ff

[benz0li]

With a 65 GB address space it works:

docker run --rm -ti denoland/deno:2.1.1 bash -c 'ulimit -v 68157440; deno'

Deno 2.1.1
exit using ctrl+d, ctrl+c, or close()
REPL is running with all permissions allowed.
To specify permissions, run `deno repl` with allow flags.
>

Of course, it also works with an unlimited address space.

[benz0li]

'Deno 1.37.2'/'v8 11.8.172.13' (bundled with Quarto 1.4.555) run out of memory when the address space is below 2 GB.

'Deno 1.44.0'/'v8 12.1.285.27' (bundled with Quarto 1.5.57) run out of memory when the address space is below 16 GB.

'Deno 1.46.3'/'v8 12.9.202.5-rusty' (bundled with Quarto 1.6.38) run out of memory when the address space is below 64 GB.

Originally posted by @benz0li in https://github.com/quarto-dev/quarto-cli/issues/4197#issuecomment-2505333958