Question: Performance Tuning

[johannesvollmer]

Hi! I'm wondering, are there some well-known steps how to achieve the best runtime performance? Which switches to flip and which patterns to embrace?

There's a lot of parameters to tweak when creating an image, but when reading an image, we have to work with the existing data. So we can't change the compression method.

What I already know:

• use more de/compression threads by specifying a thread count (best is probably num_cpus()?). does this have negative effects when the file doesn't use any compression?
• read only what you need (layers, levels, channels)

I'm not sure about:

• the original library will be built in release mode automatically, so we don't have to worry about it, right?
• scanline blocks and tiles should be written sequentially to avoid seeks, but I'm pretty sure reading the full image with one single call to file.read_pixels(0, height - 1) already does that internally, right?
• there are vectorization optimizations, are they enabled automatically?
• is there a way to disable some of the validation checks?
• is memory mapping actually faster? when reading file contents once into a buffer?

Is there more? Thanks for considering :)

[scott-wilson]

So, I think your answer can be answered in two parts:

• This is an in progress wrapper around the C++ API. We might wrap the C API instead, since that's the layer where the logic is these days. So, any questions to do with the internals of OpenEXR should be sent to the core OpenEXR project (https://github.com/AcademySoftwareFoundation/openexr).
• The Rust API is likely going to be fairly thin, and not have too much logic on the Rust side. So, likely the majority of the performance considerations are all going to be based on the C/C++ side and the communication layer. But, as for whether we'll default to release mode or not, I think that may be a question further down the line. The initial goal would be to get it working and be as close to parity as the C++ API as reasonable, then start to look at how we can make the Rust API faster. In other words, I think the initial compiler settings will be Rust's default settings, but then tuned at a later date.

[johannesvollmer]

switching to the core C API sounds reasonable. Do you think that the performance could improve by doing that? Maybe the bindings might have to go through fewer conversion steps?

[scott-wilson]

That's a good question. Honestly, I'm not too sure. But from some informal discussions with some people connected to the OpenEXR project, that seems to be the way to go in general for them. A C++ or Rust front end, and C back end. But, your guess sounds reasonable.

[johannesvollmer]

alright, thanks for the info :)