Lua bugs with unsigned integers

[GoogleCodeExporter]

Problem #1:
-- output: -1
-- expected output: 4294967295
memory.writelong(0x7E0000, -1)
print(memory.readlongunsigned(0x7E0000))

Problem #2:
-- output: a clear, red-outlined rectangle
-- expected output: a white-filled rectangle
gui.register(function()
  gui.drawbox(0,0,50,50, 0xFFFFFFFF)
end)

The first problem is a bug in readlongunsigned in practically all re-recording 
emulators. I'm guessing it's necessary to use lua_pushnumber instead of 
lua_pushinteger in that function.

The second problem seems more Snes9x-specific, and it's a somewhat recent bug 
too. 0xFFFFFFFF should become 'white' but it gets interpreted weirdly. I'm 
guessing this is because the lua51.dll included with Snes9x was compiled 
without the necessary fixes that have been made in luaconf.h in the branch 
(especially making sure that lua_number2int uses a double/long union instead of 
inline assembler code "fld" and "fistp"). If that's some official/standard 
lua51.dll we're using, well, it has that bug in it so we have to either compile 
our own non-standard one instead or go back to compiling against Lua 
statically... and maybe this bug affects all other re-recording emulators that 
use a Lua dll, if there are any others.

Original issue reported on code.google.com by nitsuja-@hotmail.com on 31 Jan 2011 at 8:07

[GoogleCodeExporter]

Another option to fix problem #2 is to make your own version of 
luaL_checkinteger and call it everywhere instead of using the one in Lua. Since 
lua_number2integer is inlined, it should work correctly if used directly. 
Although, I don't know what other bugs might also be happening because of the 
inconsistency of using a Lua that wasn't built with the same luaconf.h as 
Snes9x.

Original comment by nitsuja-@hotmail.com on 31 Jan 2011 at 5:37

[GoogleCodeExporter]

Confirmed broken in DeSmuME 0.9.7 too (both things). I'm not really sure about 
the DLL explanation I presented, but problem #2 was not a problem in DeSmuME 
0.9.6, so maybe it "imported" this bug recently from a change that was made to 
this branch of Snes9x.

Original comment by nitsuja-@hotmail.com on 4 Feb 2011 at 9:41

[GoogleCodeExporter]

I'm starting to think that these really are both the same issue and that the 
correct fix is to get any values that could be "unsigned long" integers using 
the tonumber or checknumber functions (instead of tointeger or checkinteger), 
followed by using the correct (non-default) lua_number2integer conversion macro 
(instead of letting it auto-cast the number, which would be slower). That way 
it doesn't depend on Lua to handle it and it doesn't slow things down either. 
I'll try fixing it in various emulators when I get a chance, if nobody else has 
fixed it by then.

Original comment by nitsuja-@hotmail.com on 4 Feb 2011 at 10:04

[GoogleCodeExporter]

Should be fixed in r192, at least in all the Snes9x branches I could find.

Original comment by nitsuja-@hotmail.com on 6 Feb 2011 at 6:34

• Changed state: Fixed