starwing / lua-protobuf

A Lua module to work with Google protobuf
MIT License
1.75k stars 387 forks source link

added support for luajit FFI cdata integers #77

Closed edam closed 5 years ago

edam commented 5 years ago

Added support for LuaJIT's FFI extension's wrapped integer types, so that using their 64-bit values is supported.

E.g., it should be possible to store the number 1540220658123938661 in a protobuf uint64 field. But lua <= 5.2 can not handle this number. However LuaJIT comes with the FFI extension which can wrap c types. This is used by tarantool, for example, to return 64-bit nanosecond timestamps.

lua> 1540220658123938661 -- lua 5.2 can not represent this number
1540220658123938560
lua> ffi = require( 'ffi' )
lua> x = ffi.new( 'uint64_t' ) + 154022065812393
lua> x = x * 10000 + 8661
lua> print( x, type( x ) )
1540220658123938661  cdata
lua> y = x + 1
lua> print( y, type( y ) )
1540220658123938662  cdata

This works well, but passing one of these values to lua-protobuf causes problems -- it doesn't know what to do with the cdata. Calling tonumber() (or tostring()) on it returns an (incorrect) approximation of the number (and lua-protobuf probably wouldn't like a string anyway).

This PR extends lpb_tointegerx() to use LuaJIT > 2.1, when available, to check if the cdata is an FFI-wrapped integer type that it can use.

starwing commented 5 years ago

maybe you could use the 'string int64' feature? set pb.option "int64_as_string", then pass, e.g. "#1540220658123938661" in protobuf message?

edam commented 5 years ago

After investigating a way of fixing the build problems in CI, it seems these changes are not the right approach to accessing LuaJIT FFI integers. The problem is that the code links against LuaJIT headers that are not intended for distribution (i.e., they the internal implementation -- they could even change completely between releases). The correct approach is probably something like require('ffi') and then ffi.typeof() to determine the type. But in any case, this is a completely different approach and this PR is probably invalid.