search

cwida / ALP

ALP: Adaptive Lossless Floating-Point Compression
MIT License
57 stars 7 forks source link

I fill confused about the -DALP_BUILD_GENERATED=ON #7

Closed lovedancer075 closed 2 months ago

lovedancer075 commented 2 months ago

Hi, I fill confused about the -DALP_BUILD_GENERATED=ON.

My concern is:

  1. If I want to use SIMD which means I need the fastest speed during compress and decompress, do I need to set -DALP_BUILD_GENERATED=ON in CMakeLists.txt?
  2. if DALP_BUILD_GENERATED=OFF, No SIMD is applied and the speed is lower, Am I right?

Thank you very much for your amazing lossless float compression algrithm!

azimafroozeh commented 2 months ago

Hi,

Thanks for your questions!

ALP utilizes auto-vectorization to benefit the SIMD capabilities of modern hardware. This means the core library is implemented using scalar-only code, not explicit SIMD instructions. However, it achieves the same performance as implementations based on explicit SIMD.

The -DALP_BUILD_GENERATED flag is only used to benchmark the decompression speed of explicit SIMD implementations versus scalar code. This was done to show that ALP can match the speed of explicit SIMD implementations by only using scalar-code and relying on the compiler's auto-vectorization capabilities.

Your Questions:

  1. No, you do not need to set -DALP_BUILD_GENERATED=ON to use SIMD. However, you do need to compile the ALP library with the appropriate flags. For example, on x86 with the AVX-512 instruction set, you should use the -mavx512 flag.

  2. No, setting -DALP_BUILD_GENERATED=OFF is only for benchmarking the explicit SIMD implementation of ALP. The scalar implementation can achieve the same performance with the right compiler flags.

Conclusion:

To achieve the fastest speed using SIMD during compression and decompression, make sure to compile with the correct flags that match your hardware's capabilities. We will add this feature in the future to auto-detect the right flag within the ALP library.

Please let us know if you have more questions!

lovedancer075 commented 2 months ago

Thank you for your reply. I understand it now. Sincere thanks!