Incorrect node tree on big-endian systems

[GoogleCodeExporter]

I found a problem on big-endian systems, e.g. Sun
SPARC/Solaris, where the node tree for hierarchical models
is not being built correctly.

The problem is in lib3ds_file.c, kfdata_read.  It passes the
address of a Lib3dsNode user_id to bsearch.  user_id is an
unsigned int.  But the bsearch comparison routine
compare_node_id2 is casting it to the address of an unsigned
short.  This will work on little-endian systems (for
user_id<65535) but doesn't work on a big-endian system.  When
the user_id value is accessed the top two (null) bytes are
being referenced.

The following simple patch fixes the problem.

$ diff -u lib3ds_file.c lib3ds_file.c.patch
--- lib3ds_file.c       Fri Apr 16 19:29:54 2010
+++ lib3ds_file.c.patch Mon Apr 19 15:18:05 2010
@@ -452,7 +452,7 @@

 static int 
 compare_node_id2( const void *a, const void *b ) {
-   return *((unsigned short*)a) - (*((Lib3dsNode**)b))->node_id;
+   return *((unsigned*)a) - (*((Lib3dsNode**)b))->node_id;
 }

Original issue reported on code.google.com by cjmcdona...@gmail.com on 19 Apr 2010 at 5:09

[GoogleCodeExporter]

Original comment by jkypr...@gmail.com on 19 Apr 2010 at 6:58

• Changed state: Accepted