Consider slipstream for auto-vectorization

jorgecarleitao / arrow2

Transmute-free Rust library to work with the Arrow format

Apache License 2.0

1.06k stars 221 forks source link

Open jorgecarleitao opened 3 years ago

jorgecarleitao commented 3 years ago

@Dandandan raised the idea of considering the slipstream crate for SIMD / auto-vectorization.

I agree that we should consider it - if anything, to offload that part of the code-base to slipstream. IMO our requirements are:

As secure or more secure than the existing implementation
As fast or faster than the existing implementation
compatible with the bitmaps that we use for masks, including bitmaps with offsets

jorgecarleitao commented 3 years ago

Pinging @vorner, so they are aware of this :)

vorner commented 3 years ago

It would be an honor. Just be sure to measure it, because it sometimes has a bit unreliable results :-(.