search

sile / libflate

A Rust implementation of DEFLATE algorithm and related formats (ZLIB, GZIP)
https://docs.rs/libflate
MIT License
180 stars 35 forks source link

Streaming data extensions #58

Closed codertao closed 3 years ago

codertao commented 3 years ago

I'm looking to use libflate for a long running stream of data having periodic small messages (<1k), upto 30 seconds apart.

I'm still in the early stages of investigation, but I think what I need for my use case is:

  1. a method for flushing the current block on Encoder, and (when needed), appending a zero length block to get bitwriter to the next byte boundary
  2. a new huffman type to dynamically choose between fixed and static table, depending on which would give better block size

Is that reasonable? Am I misreading anything / have I missed existing functionality?

Would there be interest in a PR for this?

sile commented 3 years ago

Thank you for reaching out. FYI, there is #28 that adds support of incremental writes to libflate. Could this PR help you?

codertao commented 3 years ago

Aye, #28 would be half of this. Any idea why that was never merged? Looks to have been inactive for the past year and a half.

sile commented 3 years ago

I was busy at that time and just had forgotten the PR 😅 I think that I can merge this PR within several days.

sile commented 3 years ago

Merged #28 and released libflate-v1.0.4.