google / xls

XLS: Accelerated HW Synthesis
http://google.github.io/xls/
Apache License 2.0
1.18k stars 173 forks source link

add DSLX sample for Dictionary Based Encoding (DBE) encoder/decoder #995

Open proppy opened 1 year ago

proppy commented 1 year ago

See https://en.wikipedia.org/wiki/Dictionary_coder

This will help demonstrate how to implement streaming algorithms that heavily rely on stateful memory with DSLX procs.

proppy commented 1 year ago

Possible algorithms researched:

rdob-ant commented 1 year ago

In addition to the above, we're also considering two other algorithms:

For me, it is tempting to go with LZ4. Due to its relatively widespread use it would make a much more "useful" DSLX sample, while still being similar to the initially considered ALDC (so most of our ALDC-related ideas are still applicable), and at the same time not resulting in much scope creep (e.g. adding Huffman or entropy coders/decoders on top of the dictionary coder - which would be required for most other algorithms).

proppy commented 1 year ago

Also learned about the following today:

rdob-ant commented 1 year ago

Opened draft #1038 which shows current state of matters