search

mirage / ocaml-solo5

Freestanding OCaml runtime
Other
100 stars 30 forks source link

dlmalloc: expose struct mallinfo mallinfo () -- malloc statistics #54

Closed hannesm closed 5 years ago

hannesm commented 5 years ago

my motivation is gathering metrics from unikernels. The OCaml runtime provides access to the garbage collector statistics, but not every byte allocated in OCaml is going through OCaml's gc (bigarray/cstruct/io-page do not). Now, dlmalloc has built-in statistics that can be enabled via a define (there are actually two, MALLINFO and MALLOC_STATS - where the latter prints statistics on stdout -- I propose only to enable MALLINFO).

What is the impact? There is no work done during each malloc or free, but instead the function mallinfo() walks over the heap and gathers a set of statistics. This means, mallinfo() is (rather) expensive to call, but there's no time or space downside for users who are not interested in mallinfo.

I added the mallinfo struct and mallinfo function to stdlib.h, where the other malloc-exported functions are. If there is a better place for these, I'm fine with this as well (maybe malloc.h?). The standard type (in dlmalloc.i) for mallinfo members is size_t (but can be overriden via a define - we don't do that).

EDIT: I have deployed this change, together with changes in mirage-solo5 that bind mallinfo () and expose it as metrics (called every 10s), reporting to influx/grafana - see https://github.com/hannesm/monitoring-experiments if you're interested.

mato commented 5 years ago

This seems reasonable on the ocaml-freestanding side but for completeness can you point me to the changes you made to mirage-solo5 to expose mallinfo()?

hannesm commented 5 years ago

mirage-solo5 changes at https://github.com/mirage/mirage-solo5/pull/43 -- you may notice this just defines stuff, but no action is done (the malloc_metrics source needs to be periodically triggered to do the statistics -- something I'm not yet sure how to do best) -- in here, the metrics_src is gathering data every 10 seconds...

hannesm commented 5 years ago

this is imho ready to be merged, I provided initial code changes for mirage-solo5. or is there more to do here @mato?

mato commented 5 years ago

Sorry for the delay. I think this is fine.