Improvements to archives handling

epwalsh / rust-cached-path

🦀 Rust utility for accessing both local and remote files through a unified interface

Apache License 2.0

29 stars 14 forks source link

Improvements to archives handling #74

Open gbrls opened 1 year ago

gbrls commented 1 year ago

Added infer as a dependency so the inference of the archive type (zip, gzip) doesn't rely on the file extension.
Added option to extract the archive to a specific directory.
The raw archive is deleted after it's extracted. (This may cause problems, in my testings the archive are downloaded again if they're deleted)

epwalsh commented 1 year ago

The raw archive is deleted after it's extracted. (This may cause problems, in my testings the archive are downloaded again if they're deleted)

Yea this probably break some assumptions, but I think we can fix that. If we keep track of the extraction path in the metadata we could avoid downloading again when the extraction path exists.

gbrls commented 1 year ago

~The redownload issue was fixed using the approach you wrote above! The extraction and deletion of the original archive seems to be working without breaking anything else now.~ (There might be some corner cases when combining the extraction_dir and subdir options).

I found a bug in my implementation, it's still redownloading the full archive.

gbrls commented 1 year ago

Makes total sense to me! I'll be back later with those changes, thank you for the reviews and guidance.

YtvwlD commented 7 months ago

This is related to #68, right? What's missing for this to work?

I was trying to download a .tar.zstd archive (but the URL has no file extension) with this crate and it would be great if it worked. :)