eyalroz / libgiddy

Giddy - A lightweight GPU decompression library
BSD 3-Clause "New" or "Revised" License
42 stars 9 forks source link

Implement compression primitives for runtime-specified-length data #9

Open eyalroz opened 7 years ago

eyalroz commented 7 years ago

Our current implementation of compression schemes (see Issues #24, #25) all assume the uncompressed data type is known at compile time. In other words, there will be a separate instantiation for each type; this has its downsides even for smaller-sized types, but it's especially inopportune for fixed-length strings, whose length one cannot make assumptions about at compile time (it's uniform between records of a DB, not between columns and schemata).

So, we need variants of at least some of those compression schemes which produce untyped bytes, and take the output length as a run-time parameter.