search

lemire / FastPFor

The FastPFOR C++ library: Fast integer compression
Apache License 2.0
849 stars 124 forks source link

64bits support for FastPFor #54

Closed xndai closed 5 years ago

xndai commented 5 years ago

Hi,

I extended the current FastPFor library to support 64bits integers. And I'd like to contribute it back. Is this something you'd like to take? @lemire

The basic idea is to make FastPFor a template class which then supports different types of integers. The fastback/unpack needs to be updated accordingly to support packing and unpacking over 32 bits.

The IntegerCODEC class will also add below need interfaces for 64bits encoding -

` virtual void encodeArray(const uint64_t in, const size_t length, uint32_t out, size_t &nvalue, EncodeMeta * meta = nullptr) ;

virtual const uint32_t decodeArray(const uint32_t in, const size_t length, uint64_t out, size_t &nvalue, const EncodeMeta meta = nullptr);`

The 64bits support is only added to CompositeCodec, FastPFor and VariableByte. For all the other codes, the above new interface would throw not implemented exception as default behavior.

If you think this is a reasonable change, I can open a PR next week.

lemire commented 5 years ago

Please do.

You have some basic tests?

xndai commented 5 years ago

Yes, I do have. I add some tests based on Google gtest. This can also be extended to support short int in the future. But currently I only have int64.

xndai commented 5 years ago

Hi @lemire , I have opened a PR, please review it. I built and tested on my Mac with clang.

lemire commented 5 years ago

Issue closed by the submitter.