golang / snappy

The Snappy compression format in the Go programming language.
BSD 3-Clause "New" or "Revised" License
1.52k stars 164 forks source link

Possible to extend output buffer to capacity when encoding/decoding? #72

Closed joewreschnig closed 11 months ago

joewreschnig commented 1 year ago

I would find the Encode and Decode functions easier to use if they would extend their dst argument to its capacity. This would let the user optimally reuse a dst buffer when processing multiple blocks, without worrying about reallocating or preallocating.

var dst []byte
src := readSomething()
for len(src) > 0 {
    dst = snappy.Encode(dst, src)
    writeSomething(dst)
    src = readSomething()
}

The changes would be pretty simple, swapping a couple len for cap calls in the functions. However strictly speaking this is a breaking change; if someone is not expecting the codec to touch anything past len(dst) they would be unpleasantly surprised. So I wanted to check if there's interest before I prepare a PR (of course also with documentation updates and a benchmark demoing/justifying the change).

klauspost commented 11 months ago

In your example you can just dst = snappy.Encode(dst[:cap(dst)], src).