Closed vlomonaco closed 6 months ago
Changes Missing Coverage | Covered Lines | Changed/Added Lines | % | ||
---|---|---|---|---|---|
tests/evaluation/test_disk_usage.py | 6 | 9 | 66.67% | ||
avalanche/evaluation/metrics/disk_usage.py | 12 | 21 | 57.14% | ||
<!-- | Total: | 18 | 30 | 60.0% | --> |
Files with Coverage Reduction | New Missed Lines | % | ||
---|---|---|---|---|
avalanche/benchmarks/datasets/cub200/cub200.py | 1 | 27.55% | ||
avalanche/benchmarks/classic/ex_model.py | 1 | 31.13% | ||
<!-- | Total: | 2 | --> |
Totals | |
---|---|
Change from base Build 9204029670: | -0.004% |
Covered Lines: | 15091 |
Relevant Lines: | 29134 |
Hello,
This PR speeds up the
DiskUsage
metric.It turns out that if you have a lot of dirs/files in the current directory (the default target for the size inspection) especially the eval mode with
disk_usage_metrics(stream=True)
becomes super slow (the metric is computed every mini-batch).In my case it takes about 4 seconds for every inspection. I use the "du" command (it improves the timing of about 4x) if you are on Linux adding a warning in case the metric compute time exceeds 0.5 seconds.
Let me know if you think I should change anything.