golang / snappy

The Snappy compression format in the Go programming language.
BSD 3-Clause "New" or "Revised" License
1.52k stars 164 forks source link

Customize encode/decode buffer size #44

Open andyxning opened 5 years ago

andyxning commented 5 years ago

Why we can not customize encode/decode buffer size when initializing a new reader or writer through a function argument.

NSQ use snappy for its message encode/decode with one connection one snappy encoder/decoder. This will consume more and more memory according to the connections number as each connection will use 64KB memory.

Also, this is more friendly for uses cases where the uncompressed data are almost all in small size.