can not decompress data by golang "compress/flate"

[suconghou]

compress by golang

main.go

package main

import (
    "bytes"
    "compress/flate"
    "os"
)

func main() {
    data, err := encode([]byte("hello,world!"))
    if err != nil {
        panic(err)
    }
    os.WriteFile("1.data", data, 0666)
}

func encode(data []byte) ([]byte, error) {
    var buf = bytes.NewBuffer(nil)
    var w, err = flate.NewWriter(buf, flate.DefaultCompression)
    if err != nil {
        return nil, err
    }
    defer w.Close()
    _, err = w.Write(data)
    if err != nil {
        return nil, err
    }
    err = w.Flush()
    return buf.Bytes(), err
}

decompress

const fflate = require('fflate');
const fs = require('fs')

const data = fs.readFileSync('1.data')

console.info(fflate.inflateSync(data))

got error unexpected EOF

[suconghou]

https://github.com/nodeca/pako/issues/226

pako 1.0.11 is ok but pako 2.0.3 also failed, it may be a golang issue, but why ?

[101arrowz]

Could you upload the 1.data file? I'd like to investigate it myself. I don't have Golang installed, so I can't really test it without an example.

EDIT: Never mind, I found the issue after installing Golang and running your code.

You can't defer the execution of w.Close() till after you get the buffer's bytes. The DEFLATE format actually has a specific marker for the final block in a data stream, meaning you need to signal to the streaming compressor that you're finished before you get the bytes.

The reason Pako 1.0 was working was that it doesn't check that there are any more blocks in the stream. fflate and Pako 2.0 throw an error if there is an unexpected end-of-file, i.e. if the file ended before it saw the final block marker in the DEFLATE stream. Here is the fixed Go code:

package main

import (
    "bytes"
    "compress/flate"
    "os"
)

func main() {
    data, err := encode([]byte("hello,world!"))
    if err != nil {
        panic(err)
    }
    os.WriteFile("1.data", data, 0666)
}

func encode(data []byte) ([]byte, error) {
    var buf = bytes.NewBuffer(nil)
    var w, err = flate.NewWriter(buf, flate.DefaultCompression)
    if err != nil {
        return nil, err
    }
        // Don't defer w.Close()
    _, err = w.Write(data)
    if err != nil {
        return nil, err
    }
        // If you Close(), you implicitly Flush() anyway
    err = w.Close()
    return buf.Bytes(), err
}

Let me know if this works for you.

[suconghou]

It works like a charm. thank you very much !