johannesvollmer / exrs

100% Safe Rust OpenEXR file library
Other
149 stars 23 forks source link

Add pixel format conversion benchmarks #174

Closed Shnatsel closed 1 year ago

Shnatsel commented 1 year ago

The parallel RLE read benchmark is highly unreliable, showing variance from 20 to 28 without any changes to the code.

This PR adds a single-threaded equivalent of that benchmark so that performance could be more accurately measured.

I also suggest removing the parallel version to avoid confusion in the future, but that's up to you and not included in this PR.

johannesvollmer commented 1 year ago

good point. great addition, and I'll think about removing, or at least clarifying, the other benchmark :)

johannesvollmer commented 1 year ago

for a more nuanced benchmark suite, we can also add another RLE benchmark with a different storage type. Right now I see only all_channels, I'd like to add a rgba variation too. Do you use the benchmarks for profiling? In that case, adding an rgba variant will give additional insights (I suspect that rgba is the most common use case of the library).

Shnatsel commented 1 year ago

So far I've been using the examples for profiling, but there often isn't a good one for what I'm looking for, so I expect I will start using benchmarks instead.

RGBA would be a great addition if it's the common case.

johannesvollmer commented 1 year ago

great, the examples should cover most use cases, and should represent real-world use cases. For profiling though, we might want to read from in-memory buffers only, such that the delay produced by reading from disk is not too prominent