Failed to read zip archives larger than 4 GB that are created by the crate

[ChieloNewctle]

Describe the bug Failed to read files with this crate inside zip archives that are also created by the crate. The error Invalid local file header shows when reading some files around 4 GB border.

To Reproduce Steps to reproduce the behavior:

1. Create a zip archive larger than 4 GB with multiple files that are smaller than 2 GB.
2. Read the files inside the archive.
3. The error will show when reading a file around 4 GB.

Expected behavior All files can be read properly.

Workaround Archives created by other zip command line tools can be read with this crate properly.

[cosmicexplorer]

Which version of the crate demonstrates this behavior? Could I ask you to provide a code snippet I can compile and run which exhibits the bug?

[ChieloNewctle]

Which version of the crate demonstrates this behavior? Could I ask you to provide a code snippet I can compile and run which exhibits the bug?

Sure thing. Here is an example using zip 2.1.3 with the dependency fixed:

[dependencies]
zip = "=2.1.3"

Archiving the zip file:

use std::{
    fs::File,
    io::{BufWriter, Read, Write},
};

use zip::{write::SimpleFileOptions, CompressionMethod, ZipWriter};

const FILE_SIZE_1GB: usize = 1024 * 1024 * 1024;
const NUM_FILES: usize = 6;

#[derive(Debug, Default)]
pub struct FakeFile<const SIZE: usize>(usize);

impl<const SIZE: usize> Read for FakeFile<SIZE> {
    fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
        let chunk_size = buf.len().min(SIZE.saturating_sub(self.0));
        buf[..chunk_size].fill(0);
        self.0 += chunk_size;
        Ok(chunk_size)
    }
}

fn main() -> zip::result::ZipResult<()> {
    let path = std::env::args_os()
        .nth(1)
        .expect("please specify file name");

    let file = File::create(path)?;
    let mut writer = ZipWriter::new(BufWriter::new(file));

    let options = SimpleFileOptions::default()
        .compression_method(CompressionMethod::Stored)
        .large_file(true);
    // the error occurs regardless of whether the `large_file` flag is set.

    for i in 0..NUM_FILES {
        writer.start_file(i.to_string(), options)?;
        std::io::copy(&mut FakeFile::<FILE_SIZE_1GB>::default(), &mut writer)?;
    }

    writer.finish()?.flush()?;

    Ok(())
}

Unpacking the zip file:

use std::{
    fs::File,
    io::{BufReader, Read},
};

use zip::ZipArchive;

const BUF_SIZE_256MB: usize = 256 * 1024 * 1024;

fn main() -> zip::result::ZipResult<()> {
    let path = std::env::args_os()
        .nth(1)
        .expect("please specify file name");

    let mut archive = ZipArchive::new(BufReader::new(File::open(path)?))?;

    let file_names = archive
        .file_names()
        .map(|x| x.to_owned())
        .collect::<Vec<_>>();

    let mut buf = vec![0; BUF_SIZE_256MB];
    for name in file_names {
        let mut file = archive.by_name(&name)?;
        let mut total = 0;
        loop {
            let num = file.read(&mut buf)?;
            if num == 0 {
                break;
            }
            total += num;
        }
        println!("{name}: {total} bytes in total");
    }

    Ok(())
}

Running unpacking:

0: 1073741824 bytes in total
1: 1073741824 bytes in total
2: 1073741824 bytes in total
3: 1073741824 bytes in total
Error: InvalidArchive("Invalid local file header")

[LoganAMorrison]

@Pr0methean Do you have a sense of where to look for the source of this error? I hit the same issue and would like to take a stab at a solution