101arrowz / fflate

High performance (de)compression in an 8kB package
https://101arrowz.github.io/fflate
MIT License
2.27k stars 79 forks source link

AsyncUnzipInflate fails on a zip file created by AsyncZipDeflate that contains a zip file #114

Closed harveylee closed 2 years ago

harveylee commented 2 years ago

How to reproduce

  1. Create a zip file using a standard tool, or zipSync.
  2. Use AsyncZipDeflate to add that zip file to a new zip file
  3. Try to unzip using AsyncUnzipInflate
const fs = require('fs')
const { zipSync, Zip, Unzip, AsyncZipDeflate, AsyncUnzipInflate } = require('fflate')

function unzip (zipPath) {
  const zipped = fs.readFileSync(zipPath)
  const unzipper = new Unzip(stream => {
    const fileWriteStream = createWriteStream(`out_${stream.name}`)
    stream.ondata = (err, chunk, final) => {
      if (err !== null) throw err
      fileWriteStream.write(chunk)
      if (final) fileWriteStream.end()
    }
    stream.start()
  })
  unzipper.register(AsyncUnzipInflate)
  unzipper.push(zipped, true)
}

function streamZipAndUnzip (inputPath) {
  const outputPath = `stream_zipped_${inputPath}.zip`
  const writeStream = fs.createWriteStream(outputPath)
  writeStream.once('finish', () => unzip(outputPath))

  const zip = new Zip((err, data, final) => {
    if (err !== null) throw err
    writeStream.write(data)
    if (final) writeStream.end()
  })

  const file = new AsyncZipDeflate(inputPath)
  zip.add(file)
  file.push(fs.readFileSync(inputPath), true)

  zip.end()
}

function zipAndUnzip (inputPath) {
  const outputPath = `zipped_${inputPath}.zip`
  const zipped = zipSync({
    [inputPath]: fs.readFileSync(inputPath)
  })
  fs.writeFileSync(outputPath, zipped)
}

// Zipping and unzipping a plain JPG seems to work
streamZipAndUnzip('truck.jpg')

// Unzipping a zip file containing another zip file - both created using an off-the-shelf tool - seems to work
unzip('double.zip')

// Using zipSync to zip a zip file and then unzipping seems to work
zipAndUnzip('truck.zip')

// Using AsyncZipDeflate to zip a zip file - fails when unzipping with "Error: invalid distance"
streamZipAndUnzip('truck.zip')

The problem

Error: invalid distance when unzipping a zip file created using AsyncZipDeflate which contains another zip file.

It seems that when unzipping the outer zip file, it also tries to unzip the inner zip file, which is surprising behaviour?

double.zip truck.jpg truck.zip

101arrowz commented 2 years ago

I think this could be related to the fact that fflate searches for the local file headers in streaming mode, but this issue should be avoidable with some logic rework.

DustinBrett commented 2 years ago

I am also getting invalid distance when trying to unzip certain zip files. I didn't notice this during development but after building/bundling it started happening. Any suggestions how I could fix this? Thanks.

DustinBrett commented 2 years ago

My issue ended up being minification in NextJS that I fixed by disabling swcMinify.

101arrowz commented 2 years ago

Fixed the SWC minification problems at some point, don't remember when but SWC minification is not an issue anymore.

DustinBrett commented 2 years ago

Great! Cause I'd forgot about that issue and started using swcMinify again anyway. :-)