search

nejckorasa / s3-stream-unzip

Lightweight Java library to manage unzipping of large files and data in AWS S3 without knowing the size beforehand and without keeping it all in memory or writing to disk.
https://nejckorasa.github.io/posts/s3-unzip/
MIT License
38 stars 8 forks source link

You cannot unzip an archive if it contains folders with files #8

Closed SergeantSergey closed 1 year ago

SergeantSergey commented 1 year ago

When unzipping an archive in which there are packs with files or subfolders with files, I encounter the following error:

Failed initialising multipart upload with uploadId null
Aborting [S3MultipartUpload uploading to <folder>/, with uploadId null due to error: {}
com.amazonaws.services.s3.model.AmazonS3Exception: Object name contains unsupported characters. (Service: Amazon S3; Status Code: 400)

Can you please tell me if the library can unzip the archive if it contains folders?

nejckorasa commented 1 year ago

Hi, thanks for opening the ticket. It would really help if you could provide some more details in order to replicate the scenario:

Looking at the exception above, could it be that the object/folder path contains unsupported character? Please see here for the characters that are not supported: AmazonS3 - Creating object key names. Is <folder>/ an actual object path or have you just redacted the path?

Uploading archives containing folders was admittedly never tested and there are no existing tests for that use case. Ideally we can replicate the scenario in a test, e.g. S3UnzipManagerTest.

I've written a quick test and it successfully uploads a folder - that is using mock s3 but so far it has proven to be quite representative to the real s3.

If you'd like, feel free to open a PR and test the scenario yourself - I'd really appreciate it! I can look into it in a couple of days.