Can we account for stack size in our reported memory usage?

[NHDaly]

If you want to know how much RSS / resident memory julia is consuming, you can get a reasonable estimate by asking for live bytes tracked by the GC + bytes allocated for codegen. However, one large missing gap is the memory that is mmapped for task stacks, only parts of which may be resident, based on how much of the stack its owning Task has used.

Can we please add accounting that measures the resident Task stack sizes once in a while and updates some internal accounting? Assuming this is expensive, we could do it only once in a while, maybe at the end of a full GC and no more frequently than 1x per minute or something?

We were thinking on linux we could probably do this by iterating all stacks and calling mincore to report how much memory is resident? And calling that from the end of a GC so that the julia threads are all still paused so it's safe to access the list of stacks?

(Is there an equivalent monitoring or observability label?)

[gbaraldi]

We have num_stack_mappings which should have how many stacks have we mapped https://github.com/JuliaLang/julia/blob/cd78e6f85aa4f3ffd26751726738e97113d1b6d8/src/gc-stacks.c#L54-L67

[d-netto]

We have num_stack_mappings which should have how many stacks have we mapped

I'm afraid this could be too inaccurate since a lot of the stack pages are probably not resident in the common case.

[gbaraldi]

Right, but I'm not sure how we can do better than that