pierrec / lz4

LZ4 compression and decompression in pure Go
BSD 3-Clause "New" or "Revised" License
878 stars 142 forks source link

Invalid magic number when decompressing. Headers not added when compressing #208

Open anand-dotworld opened 1 year ago

anand-dotworld commented 1 year ago

I have an gin api in which i compress a string using lz4 frame compression and send the result in the response to a frontend. In the javascript I fetch that api and decompress the data using lz4 package by pierrec, at that time I am getting Invalid magic number error. Can anyone help me with this?

Backend `package main

import ( "bytes" "fmt" "io"

"github.com/pierrec/lz4"

"github.com/gin-gonic/gin"

)

var fileContent = CompressBlock compresses the source buffer starting at soffet into the destination one. This is the fast version of LZ4 compression and also the default one. The size of the compressed data is returned. If it is 0 and no error, then the data is incompressible. An error is returned if the destination buffer is too small.

func main() { router := gin.Default()

// Define the API endpoint
router.POST("/api/endpoint", func(c *gin.Context) {
    toCompress := []byte(fileContent)
    // compressed := make([]byte, len(toCompress))

    result, err := compress(toCompress)
    if err != nil {
        fmt.Println(err)
    }

    c.JSON(200, gin.H{"result": result})
})

// Start the server
router.Run(":8000")

}

func compress(in []byte) ([]byte, error) { r := bytes.NewReader(in) w := &bytes.Buffer{} zw := lz4.NewWriter(w) _, err := io.Copy(zw, r) if err != nil { return nil, err } // Closing is very important if err := zw.Close(); err != nil { return nil, err } return w.Bytes(), nil } `

Frontend ` import fetch from "node-fetch"; import LZ4 from "lz4";

const apiUrl = "http://localhost:8000/api/endpoint";

const postData = async () => { try { const response = await fetch(apiUrl, { method: "POST", headers: { "Content-Type": "application/json", }, body: JSON.stringify({}), });

if (!response.ok) {
  throw new Error("Request failed with status " + response.status);
}
const result = await response.json();
// const decoded = base64ToArrayBuffer(result.result);
const buffer = Buffer.from(result.result);
const decompressed = LZ4.decode(buffer);
const decodedBuffer = Buffer.from(decompressed);
console.log(decodedBuffer.toString());

} catch (error) { console.error(error); } };

postData(); `

Error Error: Invalid magic number: 4D466941 @0 at Decoder.emit_Error (/home/anand/go/src/github.com/anand-dotworld/test/client/node_modules/lz4/lib/decoder_stream.js:64:22) at Decoder.read_MagicNumber (/home/anand/go/src/github.com/anand-dotworld/test/client/node_modules/lz4/lib/decoder_stream.js:93:8) at Decoder._main (/home/anand/go/src/github.com/anand-dotworld/test/client/node_modules/lz4/lib/decoder_stream.js:289:25) at Decoder._transform (/home/anand/go/src/github.com/anand-dotworld/test/client/node_modules/lz4/lib/decoder_stream.js:60:7) at Decoder.Transform._write (node:internal/streams/transform:184:23) at writeOrBuffer (node:internal/streams/writable:389:12) at _write (node:internal/streams/writable:330:10) at Decoder.Writable.end (node:internal/streams/writable:609:17) at Object.LZ4_uncompress [as decode] (/home/anand/go/src/github.com/anand-dotworld/test/client/node_modules/lz4/lib/decoder.js:14:10) at postData (file:///home/anand/go/src/github.com/anand-dotworld/test/client/index.js:24:30)

Thank you