lemire / FastPFor

The FastPFOR C++ library: Fast integer compression
Apache License 2.0
850 stars 124 forks source link

Rust Bindings #115

Open mwlon opened 1 month ago

mwlon commented 1 month ago

I'd like to compare against FastPFor, and I'll need it to be accessible in Rust. If no one is able to work on this soon, I could the code part of it myself. If that's the case, I think you should still be the crates.io publisher though; my interest in FastPFor may end up being ephemeral.

lemire commented 1 month ago

You are certainly invited to write wrappers in any language you'd like. If you'd like you to publish it but you do not want to maintain the result, I could maybe help find someone. Or, if you are working to solve a temporary issue, you maybe simply keep it private.

mwlon commented 1 month ago

Got it. I'll probably write the (bindgen) bindings at some point in the next few weeks and not publish them yet. If you find someone to maintain them, let me know.

djellemah commented 1 month ago

If you're wrapping c++, bindgen doesn't help much. cxx.rs works quite well.

nyurik commented 2 weeks ago

I have also been looking for either rust bindings or a rust port of this algorithm to use in the new map tile compression. A quick github search for FastPFor rust does not produce that much useful info.

I did notice lemire/simdcomp links to bitpacking crate - which might be somewhat relevant?

mwlon commented 6 days ago

Ultimately I decided not to make these myself; I largely want to use ARM, and I realized that isn't supported.

lemire commented 6 days ago

We do support ARM: https://github.com/lemire/FastPFor/pull/108

mwlon commented 6 days ago

Ah you should probably update the README then:

Hardware Requirements

We require an x64 platform.

To fully use the library, your processor should support SSSE3. This includes almost every Intel or AMD processor sold after 2006. (Note: the key schemes require merely SSE2.)

Some specific binaries will only run if your processor supports SSE4.1. They have been purely used for specific tests however.

lemire commented 6 days ago

Ah you should probably update the README then:

Done.

lemire commented 6 days ago

Note that we have CI tests under macOS (ARM-based runners).