Failing tests with unzip which is not part of zziplib

[vaukai]

What is the purpose of these six test cases? On Gentoo Linux they all fail because unzip cannot open the test files. But, unzip is not part of zziplib. So why are these tests there?

ERROR: test_59750_infozipdir_CVE_2017_5975 unzip -o 00151-zziplib-heapoverflow-__zzip_get64' returned non-zero exit status 12.

ERROR: test_59800_infozipdir_CVE_2017_5980 unzip -o 00154-zziplib-nullptr-zzip_mem_entry_new' returned non-zero exit status 12.

ERROR: test_65430 unzip -o c006-unknown-add-main' returned non-zero exit status 12.

ERROR: test_65440 unzip -o c008-main-unknown-de' returned non-zero exit status 12.

ERROR: test_65470 unzip -o 003-unknow-def-zip' returned non-zero exit status 12.

ERROR: test_65480 unzip -o 002-mem-leaks-zip' returned non-zero exit status 12.

Passing the test suite seems possible only with them disabled or removed: https://github.com/gentoo/gentoo/pull/17636/files?file-filters%5B%5D=.ebuild#diff-001a4e7ab9355e3ae408fc78bfce8846R49

[Begasus]

Some of the failing tests are visible in Haiku also, disabled them also in an open PR atm: https://github.com/haikuports/haikuports/pull/5085 Terminal output on running the tests in Haiku 32bit (same results for gcc2 and gcc8): https://paste.ubuntu.com/p/Hd2nP7qNPY/

[gdraheim]

Some tests do compare their functionality with the infozip tool atleast, and it seems that some testcases just use to to unzip-to-pipe some zip file content. So far it was not explicit that the tool is needed - but I have added a cmake "find_program" for that now. Then both "zip" and "unzip" are handed over to the testsuite.

I have also added the option to have tests just skipped but I am not sure if that is really useful in the real world. It would be better to prepare the test environment to have the tool around.

`s.s

Ran 209 tests in 12.689s

OK (skipped=37) `

[Begasus]

Thanks! Will take a look later (when I get around to it) :+1:

[vaukai]

It's still not clear to me why it needs to fail with (errors=7) so many cases of "missing xy bytes in zipfile". Do zziplib and Gentoo have different understanding of /usr/bin/unzip (which speaking for Gentoo is from http://infozip.sourceforge.net/)?

Ran 227 tests in 12.410s FAILED (errors=7, skipped=8, expected failures=5)

Or does it mean 5 of those 7 errors were expected?

[gdraheim]

The "expected failures" are extra.

Actually, I had seen recently that Ubuntu "unzip" has some problems which RHEL/Suse "unzip" does not have. That accounts to your log as well - it is not the zziplib version to fail but the operating system "unzip" program which fails. IIRC it was even a segfault so there are probably security issues in the operating system.

[vaukai]

Some more investigation of the CVE-2017-5975 test case: https://gist.github.com/vaukai/172f18a10ef187121e5d9046db62e35e

So I think the error about that file "missing 10 bytes in zipfile" should be correct and the test case should pass.