search

ponder-sh / ponder

A backend framework for crypto apps
https://ponder.sh
MIT License
613 stars 95 forks source link

feat: improved accuracy of `indexingCacheMaxBytes` #1171

Open kyscott18 opened 9 hours ago

kyscott18 commented 9 hours ago

Background

With #929, Ponder uses an in-memory cache during the historical backfill to improve the performance of indexing functions. This cache occasionally flushes to the database to avoid an out of memory error.

The cache size is determined by indexingCacheMaxBytes, which is currently computed based on os.freemem(). https://github.com/ponder-sh/ponder/blob/e6a0ab8e696bee7135e93fdc74b8e8792b341681/packages/core/src/common/options.ts#L108-L116

Description

Having a more correct (larger) cache size can significantly improve performance.

The current calculation isn't very accurate. This is reproducible by running a Ponder app with ponder dev -v and looking for the log. indexingCacheMaxBytes on a local machine should almost certainly be multiple GB.

7:29:16 PM DEBUG indexing   Using a 64 MB indexing cache

Solving this issue involves a research project into which memory and then a small pr implementing the new logic, which would be (maybe) only scoped to common/option.ts. Ideally, we can determine the memory size at which node is going to OOM and use some percentage of that as the cache size.

Notes

https://nodejs.org/api/cli.html#--max-old-space-sizesize-in-mib is what we currently use to increase memory for some hosting environments.

kyscott18 commented 9 hours ago

https://nodejs.org/api/v8.html#v8getheapspacestatistics heap_size_limit seems quite promising