search

stangelandcl / LZ4Sharp

Unmaintained port of LZ4 Compression algorithm to C#
Other
61 stars 15 forks source link

Provide stream API #5

Open friism opened 12 years ago

friism commented 12 years ago

It'd be nice to have a stream-api also. QuickLZ has wrappers that seem to do the trick (except for some length stuff missing in LZ4): http://www.quicklz.com/QuickLZCompressionStream.cs http://www.quicklz.com/QuickLZDecompressionStream.cs

stangelandcl commented 12 years ago

Cool, I'll look at it.

filmackay commented 11 years ago

yes stream based would be fantastic - it allows you to parallel process data, ie. start processing data as it comes in off the disk rather than waiting to do all the reads up front.

filmackay commented 11 years ago

see the design of other library compression classes eg.

http://msdn.microsoft.com/en-us/library/system.io.compression.deflatestream.aspx

xorxornop commented 11 years ago

Hey, I've actually already done this. Implemented with a ring buffer to keep I/O speed high. Just tested it, got 203 MB/s compression on my 2010 MacBook Pro (2x 2.66GHz i7) using Mono 2.10, so it appears speed is indeed not degraded. Now to parallelise it.

I have it in a Bitbucket repo, here: https://bitbucket.org/_zenith/lz4stream

wernight commented 11 years ago

Nice job zenith-nz, could you send a pull request so this gets into LZ4Sharp?

xorxornop commented 11 years ago

I certainly can, just be aware my stream format is not compatible with the LZ4 spec now, as I started it when no thing existed. I really only use it between .Net components, so it hasn't bothered me, but it might you.

wernight commented 11 years ago

Sounds a good enough start. Worth mentioning in the read me too. On Apr 24, 2013 10:29 PM, "zenith-nz" notifications@github.com wrote:

I certainly can, just be aware my stream format is not compatible with the LZ4 spec now, as I started it when no thing existed. I really only use it between .Net components, so it hasn't bothered me, but it might you.

— Reply to this email directly or view it on GitHubhttps://github.com/stangelandcl/LZ4Sharp/issues/5#issuecomment-16967794 .