brendan-duncan / archive

Dart library to encode and decode various archive and compression formats, such as Zip, Tar, GZip, ZLib, and BZip2.
MIT License
403 stars 140 forks source link

fix: add a listener interface to ZipDecoder to deal with large files #270

Closed akkumar closed 6 months ago

akkumar commented 1 year ago

ZipDecoder contains a 'decodeBuffer' method to deal with (reasonably !!) large files by taking in a stream.

But to make it more complete , it should have a listener interface to deal with the files one at a time as opposed to uncompressing everything in memory (and running out of it ).

Of course, it is applicable only if the zip file is large and contains a bunch of relatively smaller individual files so they can be processed without bringing all of them into memory at once.

New type added:


/// FnArchiveFile is the callback invoked when the zip decoder iterates through the archive.
typedef FnArchiveFile = void Function(ArchiveFile file);

New method onDecodeBuffer added to ZipDecoder

  void onDecodeBuffer(
    InputStreamBase input,
    FnArchiveFile fnArchiveFile, {
    bool verify = false,
    String? password,
  }) ;

The above new method is very similar to the original decodeBuffer method , except that it takes a function ( FnArchiveFile ) as the second argument.

  Archive decodeBuffer(
    InputStreamBase input, {
    bool verify = false,
    String? password,
  }) 

This should address large files to some extent.

Let me know your comments regarding the same.

brendan-duncan commented 6 months ago

I'm closing this PR. I'm in the process of finishing up a major update version, and it includes a similar callback for the archive decoders, https://github.com/brendan-duncan/archive/tree/4.0.