search

firedancer-io / firedancer

Firedancer is Jump Crypto's Solana validator software.
https://firedancer.io
Other
927 stars 168 forks source link

Make error message when `large-pages init` fails clearer #939

Closed mmcgee-jump closed 6 months ago

mmcgee-jump commented 11 months ago 
ERR     11-09 12:29:50.438220 203969 f0   main src/app/fdctl/configure/configure.c(111): large-pages ... tried to initialize but didn't do anything ... expected at least 16 gigantic pages, but there are 0
make: *** [src/app/fddev/Local.mk:23: run] Error 1

This is a reasonably commonly expected error message that means there isn't enough contiguous memory available. We should advise the user to reboot or free up memory.

SOELTH commented 11 months ago

I've also seen some of our Debian-based machines that don't allow allocating giga-pages at all unless specifically enabled in grub. May be worth mentioning as well.

hbf731eF commented 11 months ago

not all hosting providers let you change /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages at runtime.

On CentOS 9(I know not supported atm, but should be same for RHEL9)

grubby --args='default_hugepagesz=1G hugepagesz=1G hugepages=16 hugepagesz=2M hugepages=130' --update-kernel /boot/vmlinuz-5.14.0-381.el9.x86_64 you can allocate1 GB huge pages at boot time
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/virtualization_tuning_and_optimization_guide/sect-virtualization_tuning_optimization_guide-memory-huge_pages-1gb-runtime

this should reflect, what is expected from: ./build/native/gcc/bin/fdctl mem

Required Gigantic Pages: 16 Required Huge Pages: 130