zip-rs / zip-old

Zip implementation in Rust
MIT License
731 stars 204 forks source link

Append to existing archive #440

Closed TheOnAndOnlyZenomat closed 6 months ago

TheOnAndOnlyZenomat commented 6 months ago

Hello, I am trying to append to an already existing archive with files. Browsing the docs and the repo I found new_append() and according to the PR #215 it should update the central directory header and make the new files available next to the already existing files in the archive.

use std::{
    fs::{File, OpenOptions},
    io::{Read, Write},
    path::{Path, PathBuf},
    str::FromStr,
};
use zip;

fn gather_files<'a, T: Into<&'a Path>>(path: T, files: &mut Vec<PathBuf>) {
    let path: &Path = path.into();

    for entry in path.read_dir().unwrap() {
        match entry {
            Ok(e) => {
                if e.path().is_dir() {
                    gather_files(e.path().as_ref(), files);
                } else if e.path().is_file() {
                    files.push(e.path());
                }
            }
            Err(_) => todo!(),
        }
    }
}

fn main() {
    let archive = PathBuf::from_str("./archive.zip").unwrap();
    let to_archive = PathBuf::from_str("./asdf/").unwrap();

    let existing_zip = OpenOptions::new()
        .read(true)
        .write(true)
        .append(true)
        .open(&archive)
        .unwrap();
    let mut append_zip = zip::ZipWriter::new_append(existing_zip).unwrap();

    let mut files: Vec<PathBuf> = vec![];
    gather_files(to_archive.as_ref(), &mut files);

    let mut buf: Vec<u8> = vec![];
    for file in files {
        append_zip
            .start_file(file.to_string_lossy(), Default::default())
            .unwrap();

        let mut f = File::open(file).unwrap();
        f.read_to_end(&mut buf).unwrap();

        append_zip.write_all(&buf).unwrap();
    }

    append_zip.finish().unwrap();
}

Above is my test program. It should append the files from ./asdf/ to ./archive.zip. Executing it the first time works and doesn't crash. But when inspecting the archive with 7z l ./archive.zip I get a warning WARNINGS: There are data after the end of archive and the new files don't appear in the file listing. Running the above test program again it crashes with InvalidArchive("Invalid Central Directory header") on new_append(existing_zip).unwrap().

Am I using this API wrong, did I miss it's intented usage or is this a genuine bug. If the latter is the case, I'd be very happy to try and fix it (maybe with some guidance) or provide you with testcasts/do profiling and testing.

a1phyr commented 6 months ago

I am pretty sure that append(true) is the wrong thing to do here: all writes to the file will happen at the end of the file, which is probably not what zip expects.

Can you try opening the file without this option ?

Also you have to clear buf between two files.

TheOnAndOnlyZenomat commented 6 months ago

Hey, thank you. append(true) was indeed the culprit, sorry I didn't notice that myself.