srikanth-lingala / zip4j

A Java library for zip files and streams
Apache License 2.0
2.09k stars 313 forks source link

Reached end of entry, but crc verification failed #135

Closed shannah closed 4 years ago

shannah commented 4 years ago

Latest from git fails to extract zip file that appears to be fine. The error:

testExtractZipFileCRCError(net.lingala.zip4j.ExtractZipFileIT)  Time elapsed: 0.096 sec  <<< ERROR!
net.lingala.zip4j.exception.ZipException: Reached end of entry, but crc verification failed for images/linux-logo.png
    at net.lingala.zip4j.io.inputstream.ZipInputStream.verifyCrc(ZipInputStream.java:275)
    at net.lingala.zip4j.io.inputstream.ZipInputStream.endOfCompressedDataReached(ZipInputStream.java:194)
    at net.lingala.zip4j.io.inputstream.ZipInputStream.read(ZipInputStream.java:158)
    at net.lingala.zip4j.io.inputstream.ZipInputStream.read(ZipInputStream.java:127)
    at net.lingala.zip4j.tasks.AbstractExtractFileTask.unzipFile(AbstractExtractFileTask.java:66)
    at net.lingala.zip4j.tasks.AbstractExtractFileTask.extractFile(AbstractExtractFileTask.java:58)
    at net.lingala.zip4j.tasks.ExtractAllFilesTask.executeTask(ExtractAllFilesTask.java:35)
    at net.lingala.zip4j.tasks.ExtractAllFilesTask.executeTask(ExtractAllFilesTask.java:13)
    at net.lingala.zip4j.tasks.AsyncZipTask.performTaskWithErrorHandling(AsyncZipTask.java:42)
    at net.lingala.zip4j.tasks.AsyncZipTask.execute(AsyncZipTask.java:36)
    at net.lingala.zip4j.ZipFile.extractAll(ZipFile.java:437)
    at net.lingala.zip4j.ExtractZipFileIT.testExtractZipFileCRCError(ExtractZipFileIT.java:389)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:239)
    at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:242)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:137)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
    at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
    at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)

I have forked the project and added a test case here:

https://github.com/shannah/zip4j

Zip file in question: https://github.com/shannah/zip4j/blob/master/src/test/resources/test-archives/website_template.zip

Test case https://github.com/shannah/zip4j/blob/master/src/test/java/net/lingala/zip4j/ExtractZipFileIT.java#L386-L391

srikanth-lingala commented 4 years ago

Thanks for the sample zip. I was able to reproduce this issue. Will look into it.

srikanth-lingala commented 4 years ago

I wonder which tool was used to create the zip file. It slightly deviated from the zip specification. Zip4j was designed to overcome such inconsistencies, but however, there was a problem with that logic which can arise in some specific circumstances. I fixed it. I will include the fix in the next release.

Technical info related to the error (if you are interested in details or for my future reference): The entry in the zip file had a zip64 header written to it. When this entry is defined, the corresponding entries in the zip headers have to be defined as the maximum limit in the headers. For example, the compressed size in the header has to be set to set as 4294967295 when the corresponding compressed size in zip64 header is written. But this was not the case. The original length of ~ 8kb was written in the entry header. I wonder why the tool was writing zip64 entry in the first place, because this header block is only supposed to be used when any of the sizes exceeds the usual 4 byte limit, but this was not the case in any of the sizes of this entry.

shannah commented 4 years ago

The zip file was created by Ant's Zip task.

<zip destfile="src/com/client4j/publisher/website_template.zip" basedir="web_template" />

The Ant version is 1.10.4 (part of Netbeans 11.2 distribution).

srikanth-lingala commented 4 years ago

Fixed in v2.3.1 released today.

AChep commented 4 years ago

Happened with me. Unfortunately, I can not provide a zip file publicly :(

hbergmey commented 2 years ago

I am having this problem right now again with Zip4J 2.11.1. Unfortunately I am unable to provide the file as it contains confidential information of our client. 7zip does not complain about the file.

JinChangSheng commented 10 months ago

I am having this problem right now again with Zip4J 2.11.5

natewind commented 3 months ago

I have the same linked Fossify File Manager bug in 1.0.1, which uses Zip4J 2.11.5.