arturoc / nvidia-texture-tools

Automatically exported from code.google.com/p/nvidia-texture-tools
Other
0 stars 0 forks source link

Aliasing violation causes erroneous output in GCC 4.4.4 #139

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago
A user encountered a texture quality problem, in our code that uses the 
Quality_Fastest mode, when compiled with GCC 4.4.4 on Ubuntu 10.10. I 
reproduced it myself with 4.4.4 on Gentoo; the problem didn't occur with 4.3.4 
or 4.5.0.

Eventually I narrowed it down to QuickCompressDXT.cpp selectDiagonal() 
returning non-deterministic results (which was surprising). It uses 
Vector3::xy(), which is defined as:

  inline const Vector2 & Vector3::xy() const
  {
    return *(Vector2 *)this;
  }

which is a clear violation of aliasing rules.

(Unfortunately NVTT seems to compile with no warnings enabled, so GCC wasn't 
complaining "Vector.h:153: warning: dereferencing pointer ‘<anonymous>’ 
does break strict-aliasing rules" like it normally would with "-Wall".)

Fixing this to return a proper non-reference Vector2(m_x, m_y) results in 
correct behaviour. Compilers are good at inlining so this won't be less 
efficient. The other functions that do similar tricks ought to be fixed too.

Original issue reported on code.google.com by exc...@gmail.com on 6 Oct 2010 at 1:59

GoogleCodeExporter commented 8 years ago
This issue was closed by revision r1158.

Original comment by cast...@gmail.com on 6 Oct 2010 at 2:56

GoogleCodeExporter commented 8 years ago
Thanks for pointing this out. I've fixed this in trunk and will merge the fix 
in the 2.0 branch shortly.

Original comment by cast...@gmail.com on 6 Oct 2010 at 3:02

GoogleCodeExporter commented 8 years ago
This issue was closed by revision r1167.

Original comment by cast...@gmail.com on 21 Oct 2010 at 6:54