search

tugrul512bit / CompressedStringLib

Heavy weight string with compression
GNU General Public License v3.0
3 stars 0 forks source link

Todo: SIMD/vectorization for the compression logic #1

Open tugrul512bit opened 2 years ago

tugrul512bit commented 2 years ago

Idea:

Each lane of SIMD computes an interleaved path in the input. Maybe 8-chars at a time. Compiler should be able to produce masked operations automatically in case of predicted or not predicted dictionary updates (also means a lot of dictionaries in encoding/decoding). 8 dictionaries = 64kB (L2 bandwidth) or 64 dictionaries = 512kB (maybe L3 bandwidth). Any wider operations could cause operations to be as slow as volatile/RAM bandwidth.

tugrul512bit commented 2 years ago

Dictionaries will be joines as simd lanes.