Check max_map-count on start

[Stebalien]

Checklist

• [X] This is not a new feature or an enhancement to the Filecoin protocol. If it is, please open an FIP issue.
• [X] This is not a new feature request. If it is, please file a feature request instead.
• [X] This is not brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on the lotus forum and select the category as Ideas.
• [X] I have a specific, actionable, and well motivated improvement to propose.

Lotus component

• [ ] lotus daemon - chain sync
• [ ] lotus miner - mining and block production
• [ ] lotus miner/worker - sealing
• [ ] lotus miner - proving(WindowPoSt)
• [ ] lotus miner/market - storage deal
• [ ] lotus miner/market - retrieval deal
• [ ] lotus miner/market - data transfer
• [ ] lotus client
• [ ] lotus JSON-RPC API
• [ ] lotus message management (mpool)
• [X] Other

Improvement Suggestion

We should read /proc/sys/vm/max_map_count on start and print a warning if it's less than 512<<10 (8x the default) according to @ribasushi.

[Stebalien]

Motivation: We're running out of maps due to badger and the FVM.

[ribasushi]

Prior-art UX/DX from an unrelated product: https://stackoverflow.com/q/42889241

[jennijuju]

this is needed for v1.20.0 correct?

[magik6k]

Could we manage this like we do FD limit today? (https://github.com/filecoin-project/lotus/blob/b706efc33bd392d5be4f0be6679f8c5b23d98e26/lib/ulimit/ulimit.go). Probably not given that it's system wide?

For a VERY visible warning we should use the alert system - e.g. https://github.com/filecoin-project/lotus/blob/b706efc33bd392d5be4f0be6679f8c5b23d98e26/node/modules/alerts.go#L8. Alerts:

• Get printed into loud logs
• Get logged in the journal
• Are visible persistently in lotus log alerts
• If any alerts are active, are visible in lotus[-miner] info
• (may send emails / trigger configurable actions in the future)

[ribasushi]

@magik6k yeah, it's not a ulimit type of settings. The value is still per-process, but it is set globally in sysctl.

You could be smarter and combine the alert with the "count badger datafiles" feature from another ticket. Then you can derive an intelligent ballpark based on:

• one map per badger file ( both vlog and sst )
• ~15k maps for wasmtime
• ~10k maps for go just because
• multiply the entire thing by 2x for good measure

Then:

• warn if breached

[mur-me]

I want to put my two cents in: it will be better to stop lotus with non-zero exit code on the start and put message in the logs, operator will read the reason and fix it at the begging.

It will be better UX for users - failfast instead of ignoring logs message and crash afterwards.

[ribasushi]

@TippyFlitsUK I won't have capacity to work on this: assigning it to me ensures it will never get done :)