Test fails on macOS - Githubissues

catap commented 2 years ago

I'm trying to build 0.13.72 on macOS Big Sur with enabled tests and one of them is failed.

√ tmp.test_65485 % lldb -- ../../bins/unzip-mem -v ZIPfile.zip
(lldb) target create "../../bins/unzip-mem"
Current executable set to '/Users/catap/src/macports-ports/archivers/libzzip/work/build/bins/unzip-mem' (x86_64).
(lldb) settings set -- target.run-args  "-v" "ZIPfile.zip"
(lldb) process launch --environment DYLD_LIBRARY_PATH=../../zzip
Process 87690 launched: '/Users/catap/src/macports-ports/archivers/libzzip/work/build/bins/unzip-mem' (x86_64)
 Length   Method    Size  Ratio   Date   Time   CRC-32    Name
--------  ------  ------- -----   ----   ----   ------    ----
       0  stored        0   0%  09-17-21 17:31         0  d/ 
--------  ------  ------- -----                           ----
Process 87690 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_ARITHMETIC (code=EXC_I386_DIV, subcode=0x0)
    frame #0: 0x0000000100003515 unzip-mem`zzip_mem_entry_direntry_done + 211
unzip-mem`zzip_mem_entry_direntry_done:
->  0x100003515 <+211>: idivq  %rsi
    0x100003518 <+214>: movl   $0x64, %r9d
    0x10000351e <+220>: subq   %rax, %r9
    0x100003521 <+223>: movq   0x4b78(%rip), %r10        ; sum_files
Target 0: (unzip-mem) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_ARITHMETIC (code=EXC_I386_DIV, subcode=0x0)
  * frame #0: 0x0000000100003515 unzip-mem`zzip_mem_entry_direntry_done + 211
    frame #1: 0x000000010000300d unzip-mem`main + 753
    frame #2: 0x00007fff20364f3d libdyld.dylib`start + 1
(lldb)

gdraheim commented 2 years ago

According to the documentation in the test code, this test is expected to fail. If the test run stops there then it may be your platform uses another return code to indicate the error. The idea is to ensure that no new bug creeps in.

 # list the ZIPfile
  exe=self.bins("unzip-mem");
  run = shell("{chdir} {tmpdir} && ../{exe} -v {zipname}.zip".format(**locals()), returncodes = [0,-8])
  logg.error("FIXME: unzip-mem test_65485 is not solved")
  self.skipTest("FIXME: not solved")

catap commented 2 years ago

@gdraheim it was just an example on macOS 11.

The full fails looks like:

======================================================================
ERROR: test_59750_infozipdir_CVE_2017_5975 (__main__.ZZipTest)
run info-zip dir test0.zip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 1850, in test_59750_infozipdir_CVE_2017_5975
    run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 123, in shell
    raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_59750 && /opt/local/bin/unzip -o 00151-zziplib-heapoverflow-__zzip_get64' returned non-zero exit status 12.

======================================================================
ERROR: test_59800_infozipdir_CVE_2017_5980 (__main__.ZZipTest)
run info-zip dir test0.zip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 2094, in test_59800_infozipdir_CVE_2017_5980
    run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 123, in shell
    raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_59800 && /opt/local/bin/unzip -o 00154-zziplib-nullptr-zzip_mem_entry_new' returned non-zero exit status 12.

======================================================================
ERROR: test_65430 (__main__.ZZipTest)
info unzip -l $(CVE).zip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 3191, in test_65430
    run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 123, in shell
    raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_65430 && /opt/local/bin/unzip -o c006-unknown-add-main' returned non-zero exit status 12.

======================================================================
ERROR: test_65440 (__main__.ZZipTest)
info unzip -l $(CVE).zip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 3234, in test_65440
    run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 123, in shell
    raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_65440 && /opt/local/bin/unzip -o c008-main-unknown-de' returned non-zero exit status 12.

======================================================================
ERROR: test_65470 (__main__.ZZipTest)
info unzip -l $(CVE).zip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 3458, in test_65470
    run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 123, in shell
    raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_65470 && /opt/local/bin/unzip -o 003-unknow-def-zip' returned non-zero exit status 12.

======================================================================
ERROR: test_65480 (__main__.ZZipTest)
info unzip -l $(CVE).zip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 3585, in test_65480
    run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 123, in shell
    raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_65480 && /opt/local/bin/unzip -o 002-mem-leaks-zip' returned non-zero exit status 12.

======================================================================
ERROR: test_65485_list_verbose_compressed_with_directory (__main__.ZZipTest)
verbously list a zipfile containing directories
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 3665, in test_65485_list_verbose_compressed_with_directory
    run = shell("{chdir} {tmpdir} && ../{exe} -v {zipname}.zip".format(**locals()), returncodes = [0,-8])
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 123, in shell
    raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_65485 && ../../bins/unzip-mem -v ZIPfile.zip' returned non-zero exit status 136.

======================================================================
ERROR: test_65670 (__main__.ZZipTest)
info unzip -l $(CVE).zip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 3794, in test_65670
    run = shell("cd {tmpdir} && {exe} -o {filename}".format(**locals()),
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 123, in shell
    raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_65670 && /opt/local/bin/unzip -o zip_poc.zip' returned non-zero exit status 12.

======================================================================
FAIL: test_65673 (__main__.ZZipTest)
unzzip-mix -l $(CVE).zip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 3844, in test_65673
    self.assertTrue(greps(run.errors, "Invalid or incomplete"))
AssertionError: [] is not true

----------------------------------------------------------------------
Ran 227 tests in 19.051s

gdraheim commented 2 years ago

The poc parts do require a download. Do you have them?

catap commented 2 years ago

@gdraheim yep, it downloads something. Anyway, I'm trying to figure out how to overstep / fix #121 before comeback here.

catap commented 2 years ago

@gdraheim figured it out. The build / downloading log was:

make[3]: Nothing to be done for `zzip/CMakeFiles/libzzip.dir/build'.
[ 47%] Built target libzzip
/Library/Developer/CommandLineTools/usr/bin/make  -f bins/CMakeFiles/zzcat.dir/build.make bins/CMakeFiles/zzcat.dir/depend
cd /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build && /opt/local/bin/cmake -E cmake_depends "Unix Makefiles" /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72 /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/bins /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/bins /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/bins/CMakeFiles/zzcat.dir/DependInfo.cmake --color=
Consolidate compiler generated dependencies of target zzcat
/Library/Developer/CommandLineTools/usr/bin/make  -f bins/CMakeFiles/zzcat.dir/build.make bins/CMakeFiles/zzcat.dir/build
make[3]: Nothing to be done for `bins/CMakeFiles/zzcat.dir/build'.
[ 57%] Built target zzcat
/Library/Developer/CommandLineTools/usr/bin/make  -f bins/CMakeFiles/zziptest.dir/build.make bins/CMakeFiles/zziptest.dir/depend
cd /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build && /opt/local/bin/cmake -E cmake_depends "Unix Makefiles" /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72 /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/bins /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/bins /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/bins/CMakeFiles/zziptest.dir/DependInfo.cmake --color=
Consolidate compiler generated dependencies of target zziptest
/Library/Developer/CommandLineTools/usr/bin/make  -f bins/CMakeFiles/zziptest.dir/build.make bins/CMakeFiles/zziptest.dir/build
make[3]: Nothing to be done for `bins/CMakeFiles/zziptest.dir/build'.
[ 68%] Built target zziptest
/Library/Developer/CommandLineTools/usr/bin/make  -f bins/CMakeFiles/zzxorcopy.dir/build.make bins/CMakeFiles/zzxorcopy.dir/depend
cd /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build && /opt/local/bin/cmake -E cmake_depends "Unix Makefiles" /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72 /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/bins /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/bins /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/bins/CMakeFiles/zzxorcopy.dir/DependInfo.cmake --color=
Consolidate compiler generated dependencies of target zzxorcopy
/Library/Developer/CommandLineTools/usr/bin/make  -f bins/CMakeFiles/zzxorcopy.dir/build.make bins/CMakeFiles/zzxorcopy.dir/build
make[3]: Nothing to be done for `bins/CMakeFiles/zzxorcopy.dir/build'.
[ 78%] Built target zzxorcopy
/Library/Developer/CommandLineTools/usr/bin/make  -f bins/CMakeFiles/zzxorcat.dir/build.make bins/CMakeFiles/zzxorcat.dir/depend
cd /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build && /opt/local/bin/cmake -E cmake_depends "Unix Makefiles" /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72 /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/bins /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/bins /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/bins/CMakeFiles/zzxorcat.dir/DependInfo.cmake --color=
Consolidate compiler generated dependencies of target zzxorcat
/Library/Developer/CommandLineTools/usr/bin/make  -f bins/CMakeFiles/zzxorcat.dir/build.make bins/CMakeFiles/zzxorcat.dir/build
make[3]: Nothing to be done for `bins/CMakeFiles/zzxorcat.dir/build'.
[ 89%] Built target zzxorcat
/Library/Developer/CommandLineTools/usr/bin/make  -f bins/CMakeFiles/zzdir.dir/build.make bins/CMakeFiles/zzdir.dir/depend
cd /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build && /opt/local/bin/cmake -E cmake_depends "Unix Makefiles" /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72 /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/bins /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/bins /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/bins/CMakeFiles/zzdir.dir/DependInfo.cmake --color=
Consolidate compiler generated dependencies of target zzdir
/Library/Developer/CommandLineTools/usr/bin/make  -f bins/CMakeFiles/zzdir.dir/build.make bins/CMakeFiles/zzdir.dir/build
make[3]: Nothing to be done for `bins/CMakeFiles/zzdir.dir/build'.
[100%] Built target zzdir
/Library/Developer/CommandLineTools/usr/bin/make  -f test/CMakeFiles/tests.dir/build.make test/CMakeFiles/tests.dir/depend
cd /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build && /opt/local/bin/cmake -E cmake_depends "Unix Makefiles" /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72 /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/test /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/test/CMakeFiles/tests.dir/DependInfo.cmake --color=
/Library/Developer/CommandLineTools/usr/bin/make  -f test/CMakeFiles/tests.dir/build.make test/CMakeFiles/tests.dir/build
cd /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/build/test && /opt/local/bin/python3.9 /opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py --exeext= --topsrcdir=/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72 -v --downloads=ON -Z /usr/bin/zip -U /usr/bin/unzip
..........................................................................................................................................................................................INFO:test:OUT  12336 zipped    61535% 0

INFO:test:ERR 
..INFO:test:OUT  12336 defl:N    61535% 000000000
 12336 defl:N    61535% 00000000000000000000
245772K defl:N    -221% 0000000000
 2060K zipped    -38224% 00000000
 12336 stored    61535% 000000000000
 12336 stored    61535% 0000000000000000
 12336 defl:N    61535% 00000p000000000

INFO:test:ERR 
.ssssss.INFO:test:OUT 
INFO:test:ERR did not open tmp.test_65461/c005-bus-zzip_parse_root_directory: tmp.test_65461/c005-bus-zzip_parse_root_directory: Illegal byte sequence

..........WARNING:test:*136: cd tmp.test_65485 && ../../bins/unzip-mem -v ZIPfile.zip
WARNING:test:ERR: /bin/sh: line 1: 35570 Floating point exception: 8   ../../bins/unzip-mem -v ZIPfile.zip
E.xxxx....F.....x...s
======================================================================
ERROR: test_65485_list_verbose_compressed_with_directory (__main__.ZZipTest)
verbously list a zipfile containing directories
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 3665, in test_65485_list_verbose_compressed_with_directory
    run = shell("{chdir} {tmpdir} && ../{exe} -v {zipname}.zip".format(**locals()), returncodes = [0,-8])
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 123, in shell
    raise subprocess.CalledProcessError(run.returncode, sh_command, output)
subprocess.CalledProcessError: Command 'cd tmp.test_65485 && ../../bins/unzip-mem -v ZIPfile.zip' returned non-zero exit status 136.

======================================================================
FAIL: test_65673 (__main__.ZZipTest)
unzzip-mix -l $(CVE).zip
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/var/macports/build/_Users_catap_src_macports-ports_archivers_libzzip/libzzip/work/zziplib-0.13.72/test/zziptests.py", line 3844, in test_65673
    self.assertTrue(greps(run.errors, "Invalid or incomplete"))
AssertionError: [] is not true

----------------------------------------------------------------------
Ran 227 tests in 82.153s

FAILED (failures=1, errors=1, skipped=7, expected failures=5)

catap commented 2 years ago

@gdraheim ok, I understand that's going on with 65485 test.

You expects -8 but on macOS exit code on this case is 128 + |-8| = 136.

I see two way to fix it:

add 136 as expected code;
introduce inside shell case that -x can be 128 + x.

gdraheim commented 2 years ago

I'd say the latter is the better one as values beyond 128 are application-specific on other platforms, so there is hardly any overlap that could go wrong.

catap commented 2 years ago

@gdraheim here it is. Also I've tracked to cause of another tests which is failed.

It is differences on output for zip and unzip from MacPorts.

I've created separated issue to track it: https://github.com/gdraheim/zziplib/issues/123

gdraheim / zziplib

Test fails on macOS #120