file loaded using 'do $file' with an absolute path includes drive letter on Windows and cannot be unpacked

[shawnlaffan]

Summary

Steps to reproduce are below.

I hit this problem when packing a file that depended on FFI::Platypus, which loads its config.pl file using the do file syntax, with an absolute path for the filename. As documented, the file path is added to %INC as both key and value. https://perldoc.perl.org/functions/do.html

When PAR tries to unpack it, Archive::Zip::Archive throws an error due to the colon in the file name.

I am not sure of the best approach to handle it. If the file is in a perl directory structure then the path could be left-truncated so it is packed and unpacked along with the other files under inc/lib. If it is elsewhere on the system, though, then that won't work so well.

As a data point, 7-zip converts the colon to an underscore when extracting, but does not allow the folder to be renamed inside the archive.

Steps to reproduce:

(Tested with PAR::Packer 1.045, Strawberry perl 5.26.1 via berrybrew).

Generate a file called file_to_do.pl. The contents don't matter greatly:

sub nonesuch {
    return 1;
}

Then pack it using do file with the absolute path name:

pp -x -e "use Path::Tiny; my $fname = Path::Tiny::path (q{./file_to_do.pl})->absolute; do $fname; my ($key) = grep {$_ =~ /file_to_do.pl/} keys %INC; print qq{$key listed at $INC{$key}\n}"

which results in:

C:/Users/user/file_to_do.pl listed at C:/Users/user/file_to_do.pl

then call the newly created exe file:

a.exe

which results in:

mkdir C:\Users\user\AppData\Local\Temp\par-736861776e\cache-698cac0bf5bb21686699b7df22397f891e18d751\inc\lib\C:\: Invalid argument; The filename, directory name, or volume label syntax is incorrect at C:/berrybrew/5.26.1_64_PDL/perl/vendor/lib/Archive/Zip/Archive.pm line 195.

[rschupp]

Hi Shawn,

your example is contrived and the result is no surprise. It looks like there's a glitch in the handling of the -x option, most likely in function _compile_or_execute in Module/ScanDeps.pm (or how its results are incorporated by PAR::Packer). But in the above case, there's nothing we can do.

But FFI::Platypus seems to pack just fine. I tried the following script

#!/usr/bin/perl
use strict;
use warnings;
use feature qw( say );
use FFI::Platypus;

my $ffi = FFI::Platypus->new;
$ffi->lib(undef);
$ffi->attach(strcmp => ["string", "string"] => "int");

say "abc : def ", strcmp("abc", "def");
say "def : abc ", strcmp("def", "abc");
say "abc : abc ", strcmp("abc", "abc");
say "ab  : abc ", strcmp("ab",  "abc");

packed it using

$ pp -o plat.exe -M FFI::Platypus:: plat.pl

and ran it

$ .\plat.exe
abc : def -1
def : abc 1
abc : abc 0
ab  : abc -1

Inspection of plat.exe wrt FFI::Platypus shows:

$ unzip -l plat.exe | grep FFI
     6370  08-14-2018 15:47   lib/FFI/CheckLib.pm
    15185  08-14-2018 15:47   lib/FFI/Platypus.pm
      320  08-14-2018 15:47   lib/FFI/Platypus/API.pm
      598  08-14-2018 15:47   lib/FFI/Platypus/Buffer.pm
     3043  08-14-2018 15:47   lib/FFI/Platypus/Declare.pm
      258  08-14-2018 15:47   lib/FFI/Platypus/Lang/ASM.pm
      347  08-14-2018 15:47   lib/FFI/Platypus/Lang/C.pm
     5405  08-14-2018 15:47   lib/FFI/Platypus/Lang/Win32.pm
     1431  08-14-2018 15:47   lib/FFI/Platypus/Memory.pm
     2551  08-14-2018 15:47   lib/FFI/Platypus/Record.pm
      765  08-14-2018 15:47   lib/FFI/Platypus/Record/TieArray.pm
      676  08-14-2018 15:47   lib/FFI/Platypus/ShareConfig.pm
     1126  08-14-2018 15:47   lib/FFI/Platypus/Type/PointerSizeBuffer.pm
     1439  08-14-2018 15:47   lib/FFI/Platypus/Type/StringPointer.pm
        0  08-14-2018 12:10   lib/auto/FFI/Platypus/Platypus.bs
   109056  08-14-2018 12:10   lib/auto/FFI/Platypus/Platypus.xs.dll
       64  08-09-2018 09:14   lib/auto/share/dist/FFI-Platypus/README.txt
     2386  08-14-2018 12:10   lib/auto/share/dist/FFI-Platypus/config.pl

so everything is there, esp. config.pl. Actually, FFI::Platypus does the right thing by locating config.pl via File::ShareDir and the following do $file is perfectly fine. I cheated a bit: -M FFI::Platypus:: was necessary to include all the Lang::* modules. I just committed a %Preload rule for that to Module::ScanDeps.

If I use pp -x ... instead, the executable gains an extra

 lib/\MSWin32-x64-multi-thread\auto\share\dist\FFI-Platypus\config.pl

which doesn't hurt, but isn't a correct Zip member either.

[shawnlaffan]

Hello Roderich,

The example is contrived in the sense that it is a reduced test case.

Anyway, the error needs the -x switch to manifest so FFI::Platypus (or one of its deps) uses the do file method to load its config.

Using the plat.pl file from your response, I get the results below. I am assuming the warnings are unrelated.

Given that PAR::Packer includes config.pl without the full path, the simple approach might be to skip packing anything that uses a special character, or at least where the file name matches a drive path such as $pfile =~ m|^\w:[\/]|.

Regards, Shawn.

$ perl -MPAR::Packer -e "print $PAR::Packer::VERSION"
1.045
$ perl -MFFI::Platypus -e "print $FFI::Platypus::VERSION"
0.51

$ pp -x -M FFI::Platypus:: -o plat.exe plat.pl
Use of uninitialized value $Config::Config{"archname"} in pattern match (m//) at C:/berrybrew/5.26.1_64_PDL/perl/site/lib/FFI/Platypus/Buffer.pm line 14.
Name "FFI::Platypus::Memory::strdup" used only once: possible typo at C:/berrybrew/5.26.1_64_PDL/perl/site/lib/FFI/Platypus/Memory.pm line 37.
Use of uninitialized value $Config::Config{"archname"} in pattern match (m//) at C:/berrybrew/5.26.1_64_PDL/perl/site/lib/FFI/Platypus/Buffer.pm line 14.
abc : def -1
def : abc 1
abc : abc 0
ab  : abc -1

$ plat.exe
mkdir C:\Users\user\AppData\Local\Temp\par-736861776e\cache-8a63d911e3e6ae06ce433cae9d6bdcdd61ac85dd\inc\lib\C:\: Invalid argument; The filename, directory name, or volume label syntax is incorrect at C:/berrybrew/5.26.1_64_PDL/perl/site/lib/Archive/Zip/Archive.pm line 193.

$ unzip -l plat.exe | grep FFI
     2440  13/08/2018 09:14   lib/C:\berrybrew\5.26.1_64_PDL\perl\site\lib\auto\share\dist\FFI-Platypus\config.pl
     6384  15/08/2018 09:18   lib/FFI/CheckLib.pm
    15185  15/08/2018 09:18   lib/FFI/Platypus.pm
      320  15/08/2018 09:18   lib/FFI/Platypus/API.pm
      598  15/08/2018 09:18   lib/FFI/Platypus/Buffer.pm
     3043  15/08/2018 09:18   lib/FFI/Platypus/Declare.pm
      258  15/08/2018 09:18   lib/FFI/Platypus/Lang/ASM.pm
      347  15/08/2018 09:18   lib/FFI/Platypus/Lang/C.pm
     5405  15/08/2018 09:18   lib/FFI/Platypus/Lang/Win32.pm
     1431  15/08/2018 09:18   lib/FFI/Platypus/Memory.pm
     2551  15/08/2018 09:18   lib/FFI/Platypus/Record.pm
      765  15/08/2018 09:18   lib/FFI/Platypus/Record/TieArray.pm
      676  15/08/2018 09:18   lib/FFI/Platypus/ShareConfig.pm
     1126  15/08/2018 09:18   lib/FFI/Platypus/Type/PointerSizeBuffer.pm
     1439  15/08/2018 09:18   lib/FFI/Platypus/Type/StringPointer.pm
        0  13/08/2018 09:15   lib/auto/FFI/Platypus/Platypus.bs
   113152  13/08/2018 09:15   lib/auto/FFI/Platypus/Platypus.xs.dll
       64  09/08/2018 17:14   lib/auto/share/dist/FFI-Platypus/README.txt
     2440  13/08/2018 09:14   lib/auto/share/dist/FFI-Platypus/config.pl

[shawnlaffan]

This fixes the issue for my use case, but I don't think it's particularly generic.

Insert

next if $^O =~ /mswin/i and $pfile =~ m|^\w:[\\/]|;

immediately after this if condition:

https://github.com/rschupp/PAR-Packer/blob/ad6da7c95e8c8861b6901fd9f6863600a3fbb249/lib/PAR/Packer.pm#L826-L828

[rschupp]

Hi Shawn,

can you try your original problem using HEAD from https://github.com/rschupp/Module-ScanDeps

[shawnlaffan]

Thanks Roderich,

That appears to fix the issue for both the original and FFI::Platypus test cases.

Regards, Shawn.