Issues with Automatic Unpacking/Uploading of ZIP files containing many files.

[sergejzr]

What steps does it take to reproduce the issue? Upload a ZIP file containing more than 1000 files. Attempt to upload another file immediately after receiving the error message from the initial upload.

• When does this issue occur?

During the upload of ZIP files with more than 1000 files. When initiating subsequent uploads before the deletion process of the initial 1000 files is completed.

• Which page(s) does it occurs on? Dataset upload page

• What happens? When a ZIP file containing more than 1000 files is uploaded, the extraction process stops after the first 1000 files with an error message. The 1000 files are then deleted, but users can start new uploads before the deletion process is finished. This can lead to files with the same name being present, causing the system to append a “-2” to the new file names. This results in inconsistencies in documentation, errors during software compilation, and other issues.

• To whom does it occur (all users, curators, superusers)? All users - uploaders

• What did you expect to happen? The system should check the number of files in the ZIP before attempting to extract and reject the upload if it exceeds the limit. After an error, the system should wait until all files are deleted before allowing new uploads. The system maybe should not automatically rename files upon encountering a name conflict.

Which version of Dataverse are you using?

6.2

Any related open or closed issues to this bug report?

There are several issues mentioning ZIP uploads, but not this behavior exactly.

Screenshots:

Additional suggestions Introduce a checkbox allowing users to choose whether to unpack ZIP, TGZ archives, etc. Add a warning message on the upload field indicating that archives will be unpacked, and mention the .ZIP.ZIP workaround.

No matter the issue, screenshots are always welcome.

To add a screenshot, please use one of the following formats and/or methods described here:

• https://help.github.com/en/articles/file-attachments-on-issues-and-pull-requests

[pdurbin]

@sergejzr have you tried setting :ZipUploadFilesLimit? The default is 1000 files.

Please see https://guides.dataverse.org/en/6.2/installation/config.html#zipuploadfileslimit

[sergejzr]

Hi Phil, yes, we can set the limit up

[pdurbin]

@sergejzr great. Does that solve the problem?

[sergejzr]

The problem when re-upload (and automatic renaming with suffix “-2 ”) remains. The request came directly form our users. I changed the text in the upload area, adding warnings about the zip upload. This is the best I can do at the moment.

[pdurbin]

@sergejzr do you have a small zip we can use for testing? One that will reproduce the problem?

[sergejzr]

I asked the user to provide us with the file. Interestingly, I can not reproduce it anymore with my test-zip.

[cmbz]

@sergejzr please provide a sample for testing, otherwise we can't move the issue forward.