What steps will reproduce the problem?
1. Applying a Convolution filter with a threshold of 128 (for example) results
in twice the expected frequency of pixels with the value 128.
2.
3.
What is the expected output? What do you see instead?
What version of the product are you using? r1734
Please provide any additional information below.
The integer division by 'div' rounds towards zero. To demonstrate the problem
with this, consider when a threshold like 128 is then added:
The new value of 126 represents original values >-3 and <=-2
The new value of 127 represents original values >-2 and <=-1
The new value of 128 represents original values >-1 and <1
The new value of 129 represents original values >=1 and <2
The new value of 130 represents original values >=2 and <3
A partial solution is to multiply the threshold by div, and add it to r,g,b,a
before dividing by div.
A better solution is to also add half of div. For example:
var scaledThreshold = threshold * div + div / 2;
r += scaledThreshold;
r /= div;
g += scaledThreshold;
g /= div;
b += scaledThreshold;
b /= div;
a += scaledThreshold;
a /= div;
Then, with the division still rounding towards zero:
The new value of 126 represents original values >=-2.5 and <-1.5
The new value of 127 represents original values >=-1.5 and <-0.5
The new value of 128 represents original values >=-0.5 and <0.5
The new value of 129 represents original values >=0.5 and <1.5
The new value of 130 represents original values >=1.5 and <2.5
Original issue reported on code.google.com by spdenne on 4 Aug 2015 at 11:12
Original issue reported on code.google.com by
spdenne
on 4 Aug 2015 at 11:12