Limit memory usage of cached address maps

[jstarry]

Problem

https://github.com/solana-labs/solana/pull/19326 adds an in-memory cache for address map accounts. This cache is used to transform the address map indexes in new v2 transactions into full addresses which are used to load accounts for tx processing. There is no limit to the number of address map accounts and so memory usage is a big concern.

Address map accounts in the current proposal can only store up to 255 addresses and are therefore about 8KB in size. The rent exempt balance needs to be about 0.058 SOL which is not high enough to deter a malicious user from creating many of these address map accounts. Also, any SOL used to create an address map account can be retrieved by closing the address map at a later time.

Proposed Solution

Only keep address map metadata in-memory (map key, activation epoch, deactivation epoch, and num entries) so that the runtime can quickly validate the map indexes used in a transaction. But remove the list of addresses from the special cache and instead rely on the accounts index and accounts cache to quickly lookup addresses using the known size of address map metadata with the offset in account index entries to do address lookups.

Other Potential Solutions

1. Set a large deposit fee for address map creation which is either distributed to validators or burned

2. Introduce a memory fee which is charged per epoch like rent but priced appropriately to cap memory usage of accounts that need to be cached like address map accounts and stake accounts.

3. Add a hard cap on the number of address map accounts

4. Tokenize the creation of address map accounts and then distribute those tokens fairly to protocol devs and users

[ryoqun]

how about 5. do the AccountsDb magic?

https://github.com/solana-labs/solana/pull/17103#discussion_r647926831 (note, i haven't looked at the #19326)...

[jstarry]

how about 5. do the AccountsDb magic?

This is probably the best way to avoid the need to keep the full address map in memory. For address map lookups we need to do a few things:

1. Check if the address map account is initialized and owned by the address map program
2. Ensure that the address map is (still) activated for the current epoch
3. Check that the lookup index is valid (address map contains at least index + 1 entries)
4. Finally, look up the appropriate address in the address map

If we keep the small amount of metadata for address maps in-memory to do steps 1-3 quickly, but rely on the account index and account db's caching for the address lookup, the risk of DOS from memory usage would be greatly reduced.

[jstarry]

The costs of adding an address map cache look like they would be too high for the perf benefits we gain so I'm going to close that PR and instead rely on the account index and cache for "fast enough" validation

[github-actions[bot]]

This issue has been automatically locked since there has not been any activity in past 7 days after it was closed. Please open a new issue for related bugs.