search

atomicobject / heatshrink

data compression library for embedded/real-time systems
ISC License
1.31k stars 176 forks source link

Clarification #45

Closed karthikshanmugam closed 6 years ago

karthikshanmugam commented 7 years ago

Hello,

Is it possible for the expanded size to be lesser than the data that is sunk?

The compressed file is huge (32MB to 9MB), I will receive the compressed file over a network in 1KB chunks and expand to save the file.

Best regards, Karthik

silentbicycle commented 7 years ago

Yes - it's possible for the compressed data to be larger than the original input (to rephrase your question a bit). Some compression algorithms will add a header indicating whether the next chunk of data is compressed or not (or by which method(s)), but since heatshrink is running in tightly bounded memory, it can't look ahead to do that sort of analysis.

In the worst case, heatshrink can increase the size of the original input by 1/8th. It may be worth trying different settings. Varying the window size and lookahead can significantly increase the compression ratio, by making different memory usage / depth of pattern search trade-offs.

If you have a lot of example files, try compressing them with the command line tool and a variety of settings. That will give you more performance info for your specific data set.