Track memory consumption during runtime and add appropriate benchmarks

iree-org / iree

A retargetable MLIR-based machine learning compiler and runtime toolkit.

http://iree.dev/

Apache License 2.0

2.57k stars 574 forks source link

Track memory consumption during runtime and add appropriate benchmarks #15323

Open dan-garvey opened 11 months ago

dan-garvey commented 11 months ago

Request description

AFAIK, and based on conversation in #15261, we currently don't have any tools to check if a patch regresses memory consumption at runtime.

benvanik commented 10 months ago

We do have the tools/API, just no automation. Pass --print_statistics to iree-run-module/benchmark-module/etc and it'll tell you peak memory usage.

pzread commented 10 months ago

Benchmark CI (the one triggered on PR) is already using --print_statistics to dump memory consumption. We just haven't yet uploaded it to the dashboard for regression tracking

pzread commented 5 months ago

Unassigned myself as I didn't find time to work on this. The current benchmark CI is already parsing --print_statistics into the benchmark results

https://github.com/openxla/iree/blob/56541e4a9a2b34fab011f67a4e380031904dc59f/build_tools/benchmarks/common/benchmark_definition.py#L511-L524