Memory usage tests - Githubissues

salesforce / best

:trophy: Delightful Benchmarking & Performance Testing

https://opensource.salesforce.com/best

MIT License

95 stars 10 forks source link

Open nolanlawson opened 4 years ago

nolanlawson commented 4 years ago

It would be convenient if Best could measure not only CPU time, but also memory usage. I can envision two types of tests here:

Memory leaks – e.g. take a snapshot, repeat some action n times, take another snapshot, diff the two
Absolute memory usage – e.g. take a snapshot at a point in time (maybe after a GC+delay) and report it

nolanlawson commented 4 years ago

Details:

For the leak scenario, I envision taking two heapsnapshot files and comparing them, because that's the most convenient way to figure out what actually leaked
Fo the absolute memory usage scenario, I envision just adding the Chrome/Puppeteer flag that allows precise memory stats and then use window.performance.memory.

nolanlawson commented 4 years ago

Just talked with @diervo, conclusions:

adding the ability to automatically run 1 iteration of each benchmark and measure the memory before/after (using special Chrome flag to expose precise memory) is reasonable
users can create their own leak tests by writing a test where it allocates a bunch of stuff and then de-allocates, so it would expect the diff to be ~0
putting chrome in precise memory mode should not affect CPU perf
memoryLeakIterations could default to 1
might call it a "measure"