Create workaround for zip bug with large files

schuemie commented 3 years ago

There's a bug in the zip package that I don't think will ever be fixed. We should probable think about another solution that avoids this problem.

The bug is in the zip_list() function used here. Some options that come to mind:

Drop zip altogether and instead use the zip library in Java. (as for example done in DatabaseConnector)
Avoid the need to list the files in the zip file. Instead just unzip the files in a temp subfolder (of the temp folder), and rename them from there. Note that it is important to rename the files right after extracting, because it is possible the user opens the same Andromeda file twice (e.g. in different threads)

schuemie commented 3 years ago

Another option:

Use utils::unzip(zipFileName, list = TRUE) instead

ablack3 commented 3 years ago

Sorry for the delay. I think utils::unzip(zipFileName, list = TRUE) is the simplest solution and I'll make the change in the development branch.

ablack3 commented 3 years ago

This has been fixed on the develop branch and will be included in the next release. I don't have an automated test for this issue since we need a ~3GB Andromeda zip file but I have an ad-hoc manual test.

ablack3 commented 3 years ago

Fixed in v0.5.0

OHDSI / Andromeda

Create workaround for zip bug with large files #21