KhronosGroup / OpenVX-sample-impl

OpenVX sample implementation
Apache License 2.0
139 stars 47 forks source link

Signed and unsigned values comparison in PoolingKernelImpl #23

Open dvorotnev opened 4 years ago

dvorotnev commented 4 years ago

Hello! In the function PoolingKernelImpl resulting value is calculated as follows:

result = CLAMP(result / (size_x * size_y), getMinValue(fmt), getMaxValue(fmt));

But the result variable has a type int32_t, the variables size_x and size_y have a type size_t and, therefore, a result of a dividing operation has a type size_t. This value is compared in the CLAMP macro with value INT16_MIN of type int_fast32_t which is converted to a big positive value of a type size_t. And so almost all positive resulting values will be clamped to INT16_MIN.

You can try it here.

I think that the correct way to calculate the resulting value is to convert it to the type int32_t before comparison:

result = result / (size_x * size_y);
result = CLAMP(result, getMinValue(fmt), getMaxValue(fmt));