search

AcademySoftwareFoundation / OpenColorIO

A color management framework for visual effects and animation.
https://opencolorio.org
BSD 3-Clause "New" or "Revised" License
1.78k stars 452 forks source link

Underflow bugfix & fix off-by-one error in the texture height calculation #1909

Open hannes-vernooij opened 10 months ago

hannes-vernooij commented 10 months ago

It should be max index / max size because 4096 / 4096 + 1 = 2, which means that two rows are allocated when a LUT1D with a size of 4096 fits in a 1D texture of 4096 pixels.

EDIT: In the initial commit I made the mistake of not taking into account that the last value of a row is repeated in the first value of the next row, this is now fixed. I validated the result by running a shaper LUT with a size of 4096, 4097, 8191 and 8192. I then applied the transform LUT on syntheticChart.01.exr and validated a 0 pixel difference.

Underflow bugfix

This also seems to solve an existing underflow bug that appeared with a 1D LUT with a size of 8191. When the height is calculated the value is not ceiled, which results in 8191 / 4096 = 2, while it should be 3.

This caused an underflow later in the code in: unsigned long missingEntries = width * height - (unsigned long)paddedChannel.size();

where width * height is less than (unsigned long)paddedChannel.size(); causing an underflow that results in a very high value that is iterated.

linux-foundation-easycla[bot] commented 10 months ago

CLA Signed

The committers listed above are authorized under a signed CLA.