golang / snappy

The Snappy compression format in the Go programming language.
BSD 3-Clause "New" or "Revised" License
1.52k stars 164 forks source link

support API for official snappy framing #2

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
I would like to see some functions to facilitate building a framed snappy 
stream. Some functions that would be helpful:

EncodeFrame(dst, src []byte)
StartFrames(dst)
FinishFrames(dst)

Please provide any additional information below.

The official snappy repository has a document describing the framing format for 
snappy (http://code.google.com/p/snappy/source/browse/trunk/framing_format.txt) 

Original issue reported on code.google.com by jehiah on 19 Jul 2012 at 3:58

GoogleCodeExporter commented 9 years ago
I put a library for decoding the snappy stream format here:

go get github.com/glycerine/go-unsnap-stream

Original comment by J.E.A...@gmail.com on 11 Jan 2014 at 10:36

GoogleCodeExporter commented 9 years ago
I've taken an initial pass at implementing Reader/Writers for the framing 
format at https://github.com/twotwotwo/sz/ but caveat emptor to the max: there 
is no test suite, this is immature code, etc. More detail is at 
https://code.google.com/p/snappy-go/issues/detail?id=4#c1

Original comment by twotwo...@gmail.com on 7 Aug 2014 at 8:16

GoogleCodeExporter commented 9 years ago
I should have updated this a while ago. At Bitly we have been using 
https://github.com/mreiferson/go-snappystream happily for a while.

Original comment by jehiah on 7 Aug 2014 at 8:53

GoogleCodeExporter commented 9 years ago
With two days of hard work, I've implemented the framing for snappy: 
github.com/tgulacsi/sz

Then I've found this issue :)

At least the command-line client is usable (now using 
github.com/mreiferson/go-snappystream):
    go get github.com/tgulacsi/sz

Original comment by tgulacs...@gmail.com on 15 Oct 2014 at 6:46

nigeltao commented 9 years ago

There are io.Reader and io.Writer implementations for the official framing (streaming) format: https://github.com/google/snappy/blob/master/framing_format.txt

These have existed since February 2015 (https://codereview.appspot.com/199400043/) but I did not update this bug at the time.