Select AVX, SSE, or non-optimized code at runtime

kiyo-masui / bitshuffle

Filter for improving compression of typed binary data.

Other

219 stars 76 forks source link

Select AVX, SSE, or non-optimized code at runtime #41

Closed toddlipcon closed 5 years ago

toddlipcon commented 8 years ago

For cases where users are not compiling bitshuffle themself, but rather using it in a shipped product, it's difficult to know ahead of time that the target architecture will support these instruction sets.

To get optimal runtime performance, bitshuffle should auto-detect the runtime architecture and pick the appropriate algorithm, either by an indirect branch, or by using gcc's 'ifunc' feature.

kiyo-masui commented 8 years ago

This seems like an obviously good idea. However, I have no experience shipping binaries nor do I have any need to at this time. As it would take me some time to figure out what the best way to do this is, I'm not going to take it on at this time. As always, pull requests are welcome.

toddlipcon commented 7 years ago

FWIW I did this with a hack on the consuming system rather than changing bitshuffle. https://github.com/apache/kudu/commit/f10ab4a1d7ac0735890dcab6685a9fba7edac74e in case anyone's interested. So, I probably won't take the time to do it in the upstream lib.