microsoft / DirectXTex

DirectXTex texture processing library
https://walbourn.github.io/directxtex/
MIT License
1.82k stars 447 forks source link

FlipRotate bug when using TEX_FR_ROTATE180 #117

Closed walbourn closed 6 years ago

walbourn commented 6 years ago

Reported from a customer in email

I've found what I believe to be a minor bug with the DirectX::FlipRotate() function.

On line 228 of DirectXTexFlipRotate.cpp the following logic is used to decide whether the width and height should be swapped:

if (flags & (TEX_FR_ROTATE90 | TEX_FR_ROTATE270))

But because of the values for the enums doing the check like that causes the width and height to be erroneously swapped when flags is TEX_FR_ROTATE180 (which will eventually cause a DirectXTex error on e.g. line 74 of DirectXTexFlipRotate.cpp if width and height aren't equal).

There are a few different ways I can see to fix it; here's one example that is very explicit to help make the problem more clear:

if (((flags & TEX_FR_ROTATE90) == TEX_FR_ROTATE90)
|| ((flags & TEX_FR_ROTATE270) == TEX_FR_ROTATE270))

I've found what I believe to be a minor bug with the DirectX::FlipRotate() function.

On line 228 of DirectXTexFlipRotate.cpp the following logic is used to decide whether the width and height should be swapped:

if (flags & (TEX_FR_ROTATE90 | TEX_FR_ROTATE270))

But because of the values for the enums doing the check like that causes the width and height to be erroneously swapped when flags is TEX_FR_ROTATE180 (which will eventually cause a DirectXTex error on e.g. line 74 of DirectXTexFlipRotate.cpp if width and height aren't equal).

There are a few different ways I can see to fix it; here's one example that is very explicit to help make the problem more clear:

if (((flags & TEX_FR_ROTATE90) == TEX_FR_ROTATE90)
|| ((flags & TEX_FR_ROTATE270) == TEX_FR_ROTATE270))
walbourn commented 6 years ago

Fixed in this commit

walbourn commented 6 years ago

Also updated the tests