dfinity / motoko

Simple high-level language for writing Internet Computer canisters
Apache License 2.0
517 stars 97 forks source link

How to get total allocated stable memory size including regions? #4674

Closed ZenVoich closed 1 day ago

ZenVoich commented 3 months ago

I want to add stable memory benchmarks to mops.

I tried to use ExperimentalStableMemory.size() but it only changes if I call ExperimentalStableMemory.grow(10) but not Region.grow(region, 10) and I don't have access to the user code so I cannot call Region.size(regN) for each region.

I tried some rts_* methods, but I have no idea how they are changing:

rts_stable_memory_size

|           | StableMemory | Region |
| :-------- | -----------: | -----: |
| 10 pages  |          138 |    128 |
| 100 pages |            0 |      0 |
| 256 pages |          256 |    256 |

rts_logical_stable_memory_size

|           | StableMemory | Region |
| :-------- | -----------: | -----: |
| 10 pages  |          138 |    128 |
| 100 pages |            0 |      0 |
| 256 pages |          256 |    256 |

rts_memory_size

|           | StableMemory | Region |
| :-------- | -----------: | -----: |
| 10 pages  |            0 |      0 |
| 100 pages |            0 |      0 |
| 256 pages |            0 |      0 |

rts_total_allocation

|           | StableMemory | Region |
| :-------- | -----------: | -----: |
| 10 pages  |          608 |    608 |
| 100 pages |          608 |    608 |
| 256 pages |          612 |    612 |

Each cell value here is difference after and before .grow call (value after minus value before).

crusso commented 3 months ago

The raw stable memory allocated to a region is grown in 128 page chunks, that is why there is no additional growth for 100 pages after allocation of initial 10 pages.

The runtime also tries to avoid allocating metadata for using the region system until the first Region.new, in which case any ESM pages are moved into a special, hidden region and metadate tables for regions are allocated at low addresses in physical stable memory.

I don't fully remember the details (on mobile, on holiday) but ESM.size() should count the logical pages allocated by the user to ESM. Region.size(r) should count the logical pages allocated to the region r. rts_stable_memory_size() should report the physical size of canister stable memory (including metadata to record region allocations and stable variables at last upgrade). rts_logical_stable_memory_size() is, if I recall correctly, the total number of physical stable memory pages used for regions and ESM (including metadata), but excluding any additional physical pages that were allocated for temporary stable variables data during an upgrade.

However I may be misremembering and would need to consult the code when I'm back at my desk in a week or two.

Because of various optimizations, you will get different numbers if you only use ESM, only use Regions, or use a mixture of both.