haskell / criterion

A powerful but simple library for measuring the performance of Haskell code.
http://www.serpentine.com/criterion
BSD 2-Clause "Simplified" License
501 stars 86 forks source link

Add peak bytes allocated field to Measured #260

Closed tbidne closed 2 years ago

tbidne commented 2 years ago

Adds the "peak mb allocated" to the json output for #256.

I wasn't sure which "max mb" stat we wanted to take here, so I took the final endPostGC one. In my testing endPostGC, endStatsPreGC, and startStats were all the same, in any case.

  measAllocated         = diff endPostGC gcStatsBytesAllocated
, measPeakMbAllocated   = gcStatsPeakMegabytesAllocated endPostGC

Example:

{
  "reportMeasured": [
    [
      4.930255000090256e-3,
      1.1697070000000007e-3,
      11361858,
      1,
      1236872,
      3,                /* peakMbAllocated */
      1,
      189848,
      4.644875e-3,
      8.848910000000001e-4,
      2.9445099999999983e-4,
      2.93705e-4
    ]
  ],
  ...
  "reportKeys": [
    "time",
    "cpuTime",
    "cycles",
    "iters",
    "allocated",
    "peakMbAllocated",
    "numGcs",
    "bytesCopied",
    "mutatorWallSeconds",
    "mutatorCpuSeconds",
    "gcWallSeconds",
    "gcCpuSeconds"
  ]
}
tbidne commented 2 years ago

Okay, comments addressed. Thanks for the quick feedback!

tbidne commented 2 years ago

@NightMachinary would this address your request?

RyanGlScott commented 2 years ago

Excellent work—thanks again for the patch. Would you prefer to have a new Hackage release with these changes soon? If not, I'll get around to it eventually when I next upload a round of packages to Hackage.

tbidne commented 2 years ago

@RyanGlScott Thanks for merging! No rush on my account. I understand the major version bump may provoke a desire to include more such changes to avoid a lot of "churn", and if so that's fine.

I do wonder if it would be desirable to have a "prettier" way to display consumed memory than the json output, for both allocated memory and the new peak memory (perhaps @NightMachinary would agree); Scanning the json output for each individual run -- AFAIK the only way to see such data -- is not as friendly as the consolidated data in the html and csv files. In other words, criterion seems more geared towards displaying timing data than memory usage. We could imagine, for instance, text output that shows the average peak memory across all runs, in addition to the absolute max.

But that probably requires a separate issue, and I'm unable to judge how well that fits into criterion's intended usage. I am happy to work on this, but I'd need some guidance.

Anyway, thanks again for the quick feedback and merge :slightly_smiling_face:.