Open proppy opened 1 year ago
Possible algorithms researched:
In addition to the above, we're also considering two other algorithms:
For me, it is tempting to go with LZ4. Due to its relatively widespread use it would make a much more "useful" DSLX sample, while still being similar to the initially considered ALDC (so most of our ALDC-related ideas are still applicable), and at the same time not resulting in much scope creep (e.g. adding Huffman or entropy coders/decoders on top of the dictionary coder - which would be required for most other algorithms).
Also learned about the following today:
Opened draft #1038 which shows current state of matters
See https://en.wikipedia.org/wiki/Dictionary_coder
This will help demonstrate how to implement streaming algorithms that heavily rely on stateful memory with DSLX procs.