search

FileOnQ / Imaging.Heif

A C#/.NET wrapper around libheif for decoding and processing high efficiency image formats (heif, heic).
GNU Lesser General Public License v3.0
15 stars 4 forks source link

Add Baseline to Benchmarks Report #25

Open SkyeHoefling opened 3 years ago

SkyeHoefling commented 3 years ago

Description

Add a baseline benchmark to each printed table. This will show the existing benchmark compared to the PR benchmark side-by-side. The data is saved in the repository, so we just need to extract it and merge it with the existing table

SkyeHoefling commented 2 years ago

@mitchelsellers @kenny-sellers I have a couple options for us to include baseline benchmarks, which one do you like best? Please vote with the emojis below

👍 Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -
🚀 Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -
🎉 - Create 2 tables PR Results Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -
Baseline Results Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -

Please leave any comments you may have

kenny-sellers commented 2 years ago

I voted. One thought and maybe I am an odd duck, what if baseline was first and then the PR result was below? In my mind, that makes it a little easier to compare the two because naturally I expect the baseline to be first.

SkyeHoefling commented 2 years ago
Something like this? Baseline Results Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
BASELINE_PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
BASELINE_PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
BASELINE_PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
BASELINE_PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -
PR Results Method Mean Error StdDev Gen 0 Gen 1 Gen 2 Allocated Allocated native memory Native memory leak
PrimaryImage_Write 2.438 s 0.0480 s 0.0624 s - - - 256 B 222,030,124 B -
PrimaryImage_ToArray 2.379 s 0.0475 s 0.1072 s - - - 1,943,008 B 222,029,232 B -
PrimaryImage_ToSpan 2.493 s 0.0498 s 0.0731 s - - - 88 B 222,029,232 B -
PrimaryImage_ToStream 2.413 s 0.0479 s 0.1061 s - - - 1,943,072 B 222,029,248 B -