Test failures in Archive::Zip 1.68 as seen on CPAN Testers

farblos commented 4 years ago

Collecting issues in the re-designed Archive::Zip test suite.

(1) From http://www.cpantesters.org/cpan/report/9d61334c-6832-11ea-9617-2a8d3ff4b9da:

t/02_main.t line 107 (and others in that file): most likely I forgot to specify the reference zip file for the azuztok calls when desired zip64 mode == ZIP64_HEADERS

t/21_zip64.t line 112 (and others in that file): most likely some unzip's only reject zip64 mode if they see the necessary entries and attributes in the headers. So they happily grok a zip file that has the zip64 EOCD as long as it does not uses zip64 mode in the headers. So we need a separate reference file for these ... sigh.

farblos commented 4 years ago

(2) From http://cpantesters.org/cpan/report/5ab3df7f-788e-1014-983f-9aa8bbc74101:

Died at t/24_unicode_win32.t line 53.: Seems that this Korean environment cannot handle the Euro-sign character in file names without further encoding magic. Probably catch the error on open in line 53 somehow more cleverly?

(3) From http://cpantesters.org/cpan/report/c5ab3e06-6517-11ea-81a6-177c052d0de1:

Argument "2.07_02" isn't numeric in subroutine entry: Nothing we can do anything about. See https://www.nntp.perl.org/group/perl.cpan.testers.discuss/2014/04/msg3360.html.

(4) From http://cpantesters.org/cpan/report/867b0e1c-65cd-11ea-abdd-08cfc181c852:

t/22_deflated_dir.t: Solaris' unzip obviously has problems groking some zero-sized extra fields: EF block length (0 bytes) invalid (< 4). Specifying the input jar.zipas reference should help.

(5) Besides the above, we have FreeBSD 9 and 10 consistenly failing, see #74.

pmqs commented 4 years ago

Is https://travis-ci.com/github/redhotpenguin/perl-Archive-Zip/jobs/297498980 already in you list?

t/02_main.t ................... ok
===(     726;2  32/86   1/16  0/8 )=====================================Can't call method "desiredCompressionLevel" on an undefined value at /home/travis/build/redhotpenguin/perl-Archive-Zip/build_dir/lib/Archive/Zip/Archive.pm line 296.
# Looks like your test exited with 2 before it could output anything.
t/05_tree.t ................... Dubious, test returned 2 (wstat 512, 0x200)

pmqs commented 4 years ago

Not seeing any other failures (other than the one above) with the PR I've just sent that enables github actions. See https://github.com/redhotpenguin/perl-Archive-Zip/pull/76

farblos commented 4 years ago

(6) Most likely a bug I came across when chasing (in the wrong place) what @pmqs has pointed out:

ok(rename($memberName, testPath('file.txt', PATH_ZIPFILE)));
$memberName = testPath('file.txt', PATH_ZIPFILE);

The first PATH_ZIPFILE seems fishy.

(7) Regarding the issue itself:

Is https://travis-ci.com/github/redhotpenguin/perl-Archive-Zip/jobs/297498980 already in you list?

Not yet. Rather disgusting since in the shuffled/parallel tests. Will try staring at the code.

farblos commented 4 years ago

(8) Make A::Z::addTree and A::Z::addFile more robust in case of race conditions when files or directories are being removed while addTree operates. And throw some reasonable errors providing information on the offending file name.

Regarding the issue itself I think the two possible race conditions can be: a) Source directory t could possibly be modified during test runtime (rather unlikely) or b) temporary test files with extension .t could be created and removed in parallel (more likely). (Any other proposals for possible race conditions?)

For b) a likely candidate would be test t/06_update.t which packs all tests and extracts them in its temp dir. (BTW it has been running in vicinity of t/05_tree.t in the failing test run!)

So next action item:

(9) In 05_tree.t ensure that the predicate does not match any temporary files.

pmqs commented 4 years ago

In 05_tree.t ensure that the predicate does not match any temporary files.

Just noticed that in the travis runs on my branch all the tests passed. Same with the new github test suite.

farblos commented 4 years ago

(10) From http://cpantesters.org/cpan/report/70a83a32-724e-11ea-a928-9f15f2a1ffcf:

In t/03_ex.t, calling execPerl fails with error message Cwd.c: loadable library and perl binaries are mismatched (got handshake key 0xce00080, needed 0xcd00080). Well, I hope I can blame that on the system config. The include path looks strange to me containing directories from different Perl version:

  @INC:
    /etc/perl
    /usr/local/lib64/perl5/5.30.1/x86_64-linux-thread-multi
    /usr/local/lib64/perl5/5.30.1
    /usr/lib64/perl5/vendor_perl/5.30.1/x86_64-linux-thread-multi
    /usr/lib64/perl5/vendor_perl/5.30.1
    /usr/local/lib64/perl5/5.28.2
    /usr/local/lib64/perl5/5.26.2
    /usr/local/lib64/perl5
    /usr/lib64/perl5/vendor_perl/5.28.2
    /usr/lib64/perl5/vendor_perl/5.26.2
    /usr/lib64/perl5/vendor_perl
    /usr/lib64/perl5/5.30.1/x86_64-linux-thread-multi
    /usr/lib64/perl5/5.30.1

(11) From http://cpantesters.org/cpan/report/dc6c78d4-74d6-11ea-bff5-22553d253d5e:

Yet another race condition:

File exists at t/common.pm line 36.
BEGIN failed--compilation aborted at t/common.pm line 38.
Compilation failed in require at t/02_main.t line 16.
BEGIN failed--compilation aborted at t/02_main.t line 16.

Will create directory testdir not checking for any errors, as it was done before the test suire rewrite. Probably test that it existst after calling mkdir and die if that is not the case.

(12) More t/05_tree.trace conditions seen in parallel CPAN tests as well with following symptoms:

Can't cd to (./testdir/) fDk0aYxy6N: No such file or directory
 at /tmp/user/1000/build/Archive-Zip-1.68-0/blib/lib/Archive/Zip/Archive.pm line 1115.
Can't call method "desiredCompressionLevel" on an undefined value at /tmp/user/1000/build/Archive-Zip-1.68-0/blib/lib/Archive/Zip/Archive.pm line 296.

farblos commented 4 years ago

Updated all of my previous comments to get the sub-issues better organized:

Sub-issues 3 and 10 are not-a-bugs
Sub-issues 7 and 12 are dups of sub-issue 9

ipenburg commented 4 years ago

Sub-issue 1 seems to be an issue with running a MacPorts unzip package from 2009 "UnZip 6.00 of 20 April 2009, by Info-ZIP. Maintained by C. Spieler." in combination with not catching the "Need PK compat" version mismatch.

This is the output from that version of unzip given the provided zips:

/opt/local/bin/unzip -t zip64.zip Archive: zip64.zip skipping: README need PK compat. v4.5 (can do v2.1) Caution: zero files tested in zip64.zip. 1 file skipped because of unsupported compression or encoding.

/opt/local/bin/unzip -l zip64.zip Archive: zip64.zip Length Date Time Name

   36  08-10-2012 14:33   README

   36                     1 file

/opt/local/bin/zipdetails zip64.zip 0000 LOCAL HEADER #1 04034B50 0004 Extract Zip Spec 14 '2.0' 0005 Extract OS 00 'MS-DOS' 0006 General Purpose Flag 0000 [Bits 1-2] 0 'Normal Compression' 0008 Compression Method 0008 'Deflated' 000A Last Mod Time 410A7430 'Fri Aug 10 14:33:32 2012' 000E CRC 69FFE77E 0012 Compressed Length 00000024 0016 Uncompressed Length 00000024 001A Filename Length 0006 001C Extra Length 0000 001E Filename 'README' 0024 PAYLOAD ...,V(.M..QH..IU..2...<..L....r.K...

0048 CENTRAL HEADER #1 02014B50 004C Created Zip Spec 2D '4.5' 004D Created OS 03 'Unix' 004E Extract Zip Spec 2D '4.5' 004F Extract OS 00 'MS-DOS' 0050 General Purpose Flag 0000 [Bits 1-2] 0 'Normal Compression' 0052 Compression Method 0008 'Deflated' 0054 Last Mod Time 410A7430 'Fri Aug 10 14:33:32 2012' 0058 CRC 69FFE77E 005C Compressed Length FFFFFFFF 0060 Uncompressed Length FFFFFFFF 0064 Filename Length 0006 0066 Extra Length 0014 0068 Comment Length 0000 006A Disk Start 0000 006C Int File Attributes 0000 [Bit 0] 0 'Binary Data' 006E Ext File Attributes 81A40000 0072 Local Header Offset 00000000 0076 Filename 'README' 007C Extra ID #0001 0001 'ZIP64' 007E Length 0010 0080 Uncompressed Size 0000000000000024 0088 Compressed Size 0000000000000024

0090 ZIP64 END CENTRAL DIR 06064B50 RECORD 0094 Size of record 000000000000002C 009C Created Zip Spec 2D '4.5' 009D Created OS 00 'MS-DOS' 009E Extract Zip Spec 2D '4.5' 009F Extract OS 00 'MS-DOS' 00A0 Number of this disk 00000000 00A4 Central Dir Disk no 00000000 00A8 Entries in this disk 0000000000000001 00B0 Total Entries 0000000000000001 00B8 Size of Central Dir 0000000000000048 00C0 Offset to Central dir 0000000000000048

00C8 ZIP64 END CENTRAL DIR 07064B50 LOCATOR 00CC Central Dir Disk no 00000000 00D0 Offset to Central dir 0000000000000090 00D8 Total no of Disks 00000001

00DC END CENTRAL HEADER 06054B50 00E0 Number of this disk 0000 00E2 Central Dir Disk no 0000 00E4 Entries in this disk FFFF 00E6 Total Entries FFFF 00E8 Size of Central Dir FFFFFFFF 00EC Offset to Central Dir FFFFFFFF 00F0 Comment Length 0000 Done

/opt/local/bin/unzip -t zip64-eocd.zip Archive: zip64-eocd.zip testing: README OK No errors detected in compressed data of zip64-eocd.zip.

/opt/local/bin/unzip -l zip64-eocd.zip Archive: zip64-eocd.zip Length Date Time Name

   36  08-10-2012 14:33   README

   36                     1 file

/opt/local/bin/zipdetails zip64-eocd.zip 0000 LOCAL HEADER #1 04034B50 0004 Extract Zip Spec 14 '2.0' 0005 Extract OS 00 'MS-DOS' 0006 General Purpose Flag 0000 [Bits 1-2] 0 'Normal Compression' 0008 Compression Method 0008 'Deflated' 000A Last Mod Time 410A7430 'Fri Aug 10 14:33:32 2012' 000E CRC 69FFE77E 0012 Compressed Length 00000024 0016 Uncompressed Length 00000024 001A Filename Length 0006 001C Extra Length 0000 001E Filename 'README' 0024 PAYLOAD ...,V(.M..QH..IU..2...<..L....r.K...

0048 CENTRAL HEADER #1 02014B50 004C Created Zip Spec 14 '2.0' 004D Created OS 03 'Unix' 004E Extract Zip Spec 14 '2.0' 004F Extract OS 00 'MS-DOS' 0050 General Purpose Flag 0000 [Bits 1-2] 0 'Normal Compression' 0052 Compression Method 0008 'Deflated' 0054 Last Mod Time 410A7430 'Fri Aug 10 14:33:32 2012' 0058 CRC 69FFE77E 005C Compressed Length 00000024 0060 Uncompressed Length 00000024 0064 Filename Length 0006 0066 Extra Length 0000 0068 Comment Length 0000 006A Disk Start 0000 006C Int File Attributes 0000 [Bit 0] 0 'Binary Data' 006E Ext File Attributes 81A40000 0072 Local Header Offset 00000000 0076 Filename 'README'

007C ZIP64 END CENTRAL DIR 06064B50 RECORD 0080 Size of record 000000000000002C 0088 Created Zip Spec 2D '4.5' 0089 Created OS 00 'MS-DOS' 008A Extract Zip Spec 2D '4.5' 008B Extract OS 00 'MS-DOS' 008C Number of this disk 00000000 0090 Central Dir Disk no 00000000 0094 Entries in this disk 0000000000000001 009C Total Entries 0000000000000001 00A4 Size of Central Dir 0000000000000034 00AC Offset to Central dir 0000000000000048

00B4 ZIP64 END CENTRAL DIR 07064B50 LOCATOR 00B8 Central Dir Disk no 00000000 00BC Offset to Central dir 000000000000007C 00C4 Total no of Disks 00000001

00C8 END CENTRAL HEADER 06054B50 00CC Number of this disk 0000 00CE Central Dir Disk no 0000 00D0 Entries in this disk FFFF 00D2 Total Entries FFFF 00D4 Size of Central Dir FFFFFFFF 00D8 Offset to Central Dir FFFFFFFF 00DC Comment Length 0000 Done

pmqs commented 4 years ago

@ipenburg This line below from your MacPorts unzip suggests that it doesn't support Zip64 at all.

skipping: README need PK compat. v4.5 (can do v2.1)

Version 4.5 is the version where Zip64 was added to the zip spec. Below is from APPNOTE.TXT

    4.4.3.2 Current minimum feature versions are as defined below:

     1.0 - Default value
     1.1 - File is a volume label
     2.0 - File is a folder (directory)
     2.0 - File is compressed using Deflate compression
     2.0 - File is encrypted using traditional PKWARE encryption
     2.1 - File is compressed using Deflate64(tm)
     2.5 - File is compressed using PKWARE DCL Implode 
     2.7 - File is a patch data set 
     4.5 - File uses ZIP64 format extensions
     ...

redhotpenguin / perl-Archive-Zip

Test failures in Archive::Zip 1.68 as seen on CPAN Testers #75