alexcrichton / tar-rs

Tar file reading/writing for Rust
https://docs.rs/tar
Apache License 2.0
616 stars 178 forks source link

Add `Archive::into_entries` for an iterator that owns the underlying `Read` object #325

Open orottier opened 1 year ago

orottier commented 1 year ago

For a niche use-case I found myself in need of a tar entries iterator that was Send + 'static.

I wrote a proof of concept for Archive::into_entries() at https://github.com/alexcrichton/tar-rs/compare/main...orottier:tar-rs:master

Do you think it makes sense to include this work in your library, or is it better to publish it as a forked crate? It will obviously need some more work (support for extensions, user settings set_ignore_zeros etc).

The following is now possible:

let input = File::open("tests/archives/reading_files.tar").unwrap();
let mut entries = Archive::new(input).into_entries();

// test 1: move the entry into a scoped thread
{
    let mut a = entries.next().expect("expected entry a present").unwrap();

    thread::scope(|s| {
        s.spawn(|| {
            assert_eq!(&*a.header.path_bytes(), b"a");
            let mut s = String::new();
            a.read_to_string(&mut s).unwrap();
            assert_eq!(s, "a\na\na\na\na\na\na\na\na\na\na\n");
        });
    });
}

// test 2: move the entries iterator to a new thread
thread::scope(|s| {
    s.spawn(|| {
        let mut b = entries.next().expect("expected entry b present").unwrap();
        assert_eq!(&*b.header.path_bytes(), b"b");
        let mut s = String::new();
        b.read_to_string(&mut s).unwrap();
        assert_eq!(s, "b\nb\nb\nb\nb\nb\nb\nb\nb\nb\nb\n");
    });
});

Also, because the entries mutably borrow the SimpleEntries object, it is impossible to consume the entries out of order.

Curious to hear what you think!

juanfrnz commented 2 months ago

I think it is not so niche use-case, unless i am missing something i found the same need to iterate on tar entries from an axum handler.

orottier commented 2 months ago

Hey @juanfrnz the compare link above stopped working, it should be https://github.com/alexcrichton/tar-rs/compare/main...orottier:tar-rs:master

Hope it helps! I'm not working on it anymore myself so I won't put it into a separate crate myself.