search

microsoft / DirectXTex

DirectXTex texture processing library
https://walbourn.github.io/directxtex/
MIT License
1.79k stars 441 forks source link

Conversion to FP16 saturates large values to NaNs #91

Closed elasota closed 6 years ago

elasota commented 6 years ago

To reproduce, download this image: https://hdrihaven.com/hdri/download.php?h=simons_town_rocks&r=1k Run texconv.exe -y -m 1 -f FP16 simons_town_rocks_1k.hdr

Offset 1717440 in the result DDS file is "a4 7c" which is a NaN. If WIC usage is forced off by forcing "usewic" to be false, then it produces "ff 7b" instead, the largest valid FP16 value.

Converting to FP32 produces a value of 76032.0 for that value, which is too large for FP16, but "a4 7c" is what would be 76032.0 if the exponent didn't make it a NaN, so I guess WIC isn't handling FP16 NaNs correctly?

This also happens if the image is converted to FP32, and then to FP16, so it isn't specific to this input format.

walbourn commented 6 years ago

This does sound like a WIC issue. I'm checking with the team to see if they have a clear statement of what is supposed to happen when you convert 'out of range' values...

elasota commented 6 years ago

Float truncation should at least produce the max allowed value, or infinity, not NaN. If infinity, it'd be nice to have a flag or function in DirectXTex to cap the infinities at the max value.

walbourn commented 6 years ago

The UseWICConversion function tries to catch the "where WIC does the wrong thing for textures". It looks like I just need some more cases there for going from FP32 to FP16 formats.

walbourn commented 6 years ago

See this pull request