search

msgpack / msgpack-perl

MessagePack serializer implementation for Perl / msgpack.org[Perl]
http://msgpack.org/
Other
51 stars 22 forks source link

Support IEEE 754 #31

Open akotlar opened 8 years ago

akotlar commented 8 years ago

Single precision floats defined in msgpack spec, doesn't seem to be implemented here, except during decoding stage.

This would save a significant amount of data for our team ~40% of ~80G per database.

syohex commented 8 years ago

Perl support only double type(NV) for floating point number. We can have float value as pointer type but it requires much bytes(float + pointer) than double.

akotlar commented 8 years ago

Perl has the ability to pack / unpack 32 bit floats. We don't particularly care about whether perl will store a single precision float as double precision, what we care about is that msgpack doesn't use up 44% more space to store a single precision float. Also, message pack's goal is interoperability, else we could just pack / unpack in perl for every type and save the overhead of encoding the msgpack schema.

You support unpacking single precision floats in you PP package. I propose we add packing support for the same.

Even in the pure perl implementation it would be trivial to check whether a value fits in 32 bits, right (partial implementation):

if( unpack('f', pack('f', $num) ) == $num) { pack('f', $num) } else { pack('d', $num) }

I'd be happy to submit a pull request for the pure perl version.

If the runtime packing cost seems high, why don't we offer it as an option, much like utf8, prefer_integer, or your big int support.

akotlar commented 6 years ago

Any chance you could take another look at this? Treating all floats as doubles means that our database is substantially larger than needed.