Y to ARGB functions are not rounding perfectly, and they could

[GoogleCodeExporter]

The Y to ARGB functions currently have up to one ULP of error and this could be 
reduced to zero at no cost. They should be changed to have tighter error bounds 
and the checks in the tests should be tightened to enforce this.

While examining YToARGBRow_AVX2 I created a spreadsheet to allow for easy 
experimentation with different constants to see if all rounding errors could be 
eliminated. Since both constants were >8 bits and all inputs and outputs were 
<=8 bits I felt this should be possible.

This experimentation showed that replacing 1197, 19071 with 1160, 18997 gave 
results that perfectly matched the rounded floating-point results.

Since we can get perfect results at no extra cost we should do so.

The spreadsheet is here:

https://drive.google.com/open?id=1PtJedJ_JFL-5OrRTe_2mSelCwHCl-ot-JodFhEmjV9c&au
thuser=0

The accuracy of the 'Correct' column should, of course, be double-checked to 
verify that it is indeed the correct target.

These changes may allow improvement of YUV to RGB functions also since having 
correct values for Y will give a more robust starting point.

Original issue reported on code.google.com by brucedaw...@chromium.org on 3 Feb 2015 at 10:44

[GoogleCodeExporter]

Fixed in r1259

Was
hist            -3      -2      -1      0               1       2       3
red             0       0       1809408 13140736        1827072 0       0
green           0       0       1679912 13471329        1625975 0       0
blue            168448  994816  1876480 10655488        1893376 1006336 182272

Now
hist            -3      -2      -1      0               1       2       3
red             0       0       558848  15632128        586240  0       0
green           0       0       209907  16350588        216721  0       0
blue            14848   642816  1989376 11363328        2053120 695040  18688

Original comment by fbarch...@google.com on 4 Feb 2015 at 7:53

• Changed state: Fixed