amaze_demosaic_RT.cc truncation of double to float warning

[Benitoite]

@heckflosse etc, With the new AppleClang on macOS13ß, I'm getting an ominous truncation warning from the big change in precision.

/Users/rb/art/rtengine/amaze_demosaic_RT.cc:764:67: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value [-Wabsolute-value]
                        if ((0.5 - varwt) * (0.5 - diffwt) > 0 && fabsf(0.5 - diffwt) < fabsf(0.5 - varwt)) {
                                                                  ^
/Users/rb/art/rtengine/amaze_demosaic_RT.cc:764:67: note: use function 'std::abs' instead
                        if ((0.5 - varwt) * (0.5 - diffwt) > 0 && fabsf(0.5 - diffwt) < fabsf(0.5 - varwt)) {
                                                                  ^~~~~
                                                                  std::abs
/Users/rb/art/rtengine/amaze_demosaic_RT.cc:764:89: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value [-Wabsolute-value]
                        if ((0.5 - varwt) * (0.5 - diffwt) > 0 && fabsf(0.5 - diffwt) < fabsf(0.5 - varwt)) {
                                                                                        ^
/Users/rb/art/rtengine/amaze_demosaic_RT.cc:764:89: note: use function 'std::abs' instead
                        if ((0.5 - varwt) * (0.5 - diffwt) > 0 && fabsf(0.5 - diffwt) < fabsf(0.5 - varwt)) {
                                                                                        ^~~~~
                                                                                        std::abs
/Users/rb/art/rtengine/amaze_demosaic_RT.cc:1232:29: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value [-Wabsolute-value]
                        if (fabsf(0.5 - pmwt[indx1]) < fabsf(0.5 - pmwtalt)) {
                            ^
/Users/rb/art/rtengine/amaze_demosaic_RT.cc:1232:29: note: use function 'std::abs' instead
                        if (fabsf(0.5 - pmwt[indx1]) < fabsf(0.5 - pmwtalt)) {
                            ^~~~~
                            std::abs
/Users/rb/art/rtengine/amaze_demosaic_RT.cc:1232:56: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value [-Wabsolute-value]
                        if (fabsf(0.5 - pmwt[indx1]) < fabsf(0.5 - pmwtalt)) {
                                                       ^
/Users/rb/art/rtengine/amaze_demosaic_RT.cc:1232:56: note: use function 'std::abs' instead
                        if (fabsf(0.5 - pmwt[indx1]) < fabsf(0.5 - pmwtalt)) {
                                                       ^~~~~
                                                       std::abs
/Users/rb/art/rtengine/amaze_demosaic_RT.cc:1300:29: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value [-Wabsolute-value]
                        if (fabsf(0.5 - pmwt[indx >> 1]) < fabsf(0.5 - hvwt[indx >> 1]) ) {
                            ^
/Users/rb/art/rtengine/amaze_demosaic_RT.cc:1300:29: note: use function 'std::abs' instead
                        if (fabsf(0.5 - pmwt[indx >> 1]) < fabsf(0.5 - hvwt[indx >> 1]) ) {
                            ^~~~~
                            std::abs
/Users/rb/art/rtengine/amaze_demosaic_RT.cc:1300:60: warning: absolute value function 'fabsf' given an argument of type 'double' but has parameter of type 'float' which may cause truncation of value [-Wabsolute-value]
                        if (fabsf(0.5 - pmwt[indx >> 1]) < fabsf(0.5 - hvwt[indx >> 1]) ) {
                                                           ^
/Users/rb/art/rtengine/amaze_demosaic_RT.cc:1300:60: note: use function 'std::abs' instead
                        if (fabsf(0.5 - pmwt[indx >> 1]) < fabsf(0.5 - hvwt[indx >> 1]) ) {
                                                           ^~~~~
                                                           std::abs

[ComputerNerd]

I think the code is slightly wrong. The constants should be suffixed with an f if it is to be a float constant. Otherwise the constant will be a double and it will cause the float to be implicitly cast to a double creating a reduction in performance. So 0.5 should be written as 0.5f whenever used with a float.

I'd recommend building RawTherapee with the -Wdouble-promotion flag. Both Clang and GCC support this.

[Lawrence37]

@Benitoite That looks like ART code. It is fixed in RawTherapee with 80c3eb6ac38b2092e68df0450faf5fe3ae72646b. https://github.com/Beep6581/RawTherapee/blob/dd0054918bc88d58176b3091a143e7e917bdc6fd/rtengine/amaze_demosaic_RT.cc#L778