search

rschupp / PAR-Packer

(perl) Generate stand-alone executables, perl scripts and PAR files https://metacpan.org/pod/PAR::Packer
Other
48 stars 13 forks source link

PAR::Packer and MSVC #53

Closed thoke closed 2 years ago

thoke commented 2 years ago

I've got a perl (5.32.1) built using MSVC (Visual Studio 2019). I'm now attempting to leverage that perl build and 'PAR::Packer'. Once I install all the required modules and pack a script, it runs fine on the build host. As soon as I take it to a host which no longer has the required runtime, I get a pop-up titled “hello.exe – System Error” with the message “The code execution cannot proceed because VCRUNTIME140.dll was not found. Reinstalling the program may fix this problem.”.

I've tried to use the -l option to explicitly include the shared libs (DLLs), but that has not helped.

I've also tried applying the patch below, but that has not helped either.

I'm looking for assistance in how to proceed to make sure PAR::Packer will allow me to create a portable exe.

Patch:

diff --git a/myldr/find_files_to_embed/guess.pl b/myldr/find_files_to_embed/guess.pl
index a86995b..45d28c1 100644
--- a/myldr/find_files_to_embed/guess.pl
+++ b/myldr/find_files_to_embed/guess.pl
@@ -20,7 +20,7 @@ sub find_files_to_embed
     # in the same way as libperl*.dll itself, otherwise a packed executable
     # won't run when libgcc_*.dll isn't installed.
     # The same holds for libstdc++*.dll (e.g. Strawberry Perl 5.16).
-    my ($libgcc, $libstdcpp, $libwinpthread);
+    my ($libgcc, $libstdcpp, $libwinpthread, @msvclibs);
     if ($^O eq 'MSWin32'
         and defined $Config{gccversion}             # gcc version >= 4.x was used
         and $Config{gccversion} =~ m{\A(\d+)}ms && $1 >= 4) {
@@ -30,9 +30,21 @@ sub find_files_to_embed
     if ($ld =~ /(\b|-)g\+\+(-.*)?(\.exe)?$/) {      # g++ was used to link
         $libstdcpp = find_dll("libstdc++*.$Config{so}");
     }
+    if ($ld eq 'link') {      # link was used to link, probably using MSVC, add the VCRUNTIME
+        print STDERR "ld is [$ld], finding VCRUNTIME DLLs\n";
+        for my $dll ( qw( concrt140
+            msvcp140
+            vcruntime140
+            vcruntime140_1 ) ) {
+
+            push @msvclibs, find_dll("$dll*.$Config{so}");
+        }
+        print STDERR "Found MSVC DLLs: ", join(', ', @msvclibs ), "\n";
+    }
+

     return { map { basename($_) => $_ }
-                 grep { defined } $libperl, $libgcc, $libwinpthread, $libstdcpp };
+                 grep { defined } $libperl, $libgcc, $libwinpthread, $libstdcpp, @msvclibs };
 }

Simple hello.pl script:

#!/usr/bin/env perl

use strict;
use warnings;

print "Hello, World!\n";

I've installed my modules in a local lib, so my pp command is run as follows:

perl -Iextlib-MSWin32-x64-multi-thread\lib\perl5 extlib-MSWin32-x64-multi-thread\bin\pp -o hello.exe hello.pl

Here are the dependencies according to dumpbin:

Z:\par-packer-testing>"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\dumpbin.exe" /DEPENDENTS hello.exe
Microsoft (R) COFF/PE Dumper Version 14.28.29334.0
Copyright (C) Microsoft Corporation.  All rights reserved.

Dump of file hello.exe

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    KERNEL32.dll
    ADVAPI32.dll
    VCRUNTIME140.dll
    api-ms-win-crt-runtime-l1-1-0.dll
    api-ms-win-crt-string-l1-1-0.dll
    api-ms-win-crt-filesystem-l1-1-0.dll
    api-ms-win-crt-stdio-l1-1-0.dll
    api-ms-win-crt-environment-l1-1-0.dll
    api-ms-win-crt-heap-l1-1-0.dll
    api-ms-win-crt-process-l1-1-0.dll
    api-ms-win-crt-math-l1-1-0.dll
    api-ms-win-crt-locale-l1-1-0.dll

  Summary

      3E1000 .data
        1000 .pdata
        2000 .rdata
        1000 .reloc
        3000 .rsrc
        4000 .text

By comparison, a similarly built hello-sp.exe using Strawberry Perl (happened to have 5.32.0.1 on this system), I see the following dependencies listed below and that file executes not only on the "build" host, but also another where SP is NOT installed.

Z:\par-packer-testing>"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\dumpbin.exe" /DEPENDENTS hello-sp.exe
Microsoft (R) COFF/PE Dumper Version 14.28.29334.0
Copyright (C) Microsoft Corporation.  All rights reserved.

Dump of file hello-sp.exe

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    msvcrt.dll
    ADVAPI32.dll
    KERNEL32.dll

  Summary

        1000 .CRT
        A000 .bss
      4D4000 .data
        1000 .idata
        1000 .pdata
        1000 .rdata
        3000 .rsrc
        A000 .text
        1000 .tls
        1000 .xdata
rschupp commented 2 years ago

Z:\par-packer-testing>"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\bin\Hostx64\x64\dumpbin.exe" /DEPENDENTS hello.exe Dump of file hello.exe ... Image has the following dependencies: ... VCRUNTIME140.dll

That means you're fucked. hello.exe or actually myldr/boot.exe is a very simple C program that shouldn't need linking with these DLLs. Try tweaking it's link step to get rid of them.

thoke commented 2 years ago

Crap, I was afraid of that exact situation. Thanks for the quick response. I'll look at myldr/boot.{c,exe} to see if we can remove that limitation.

thoke commented 2 years ago

Here's the Makefile target for boot.exe:

boot.exe: boot.obj sha1.obj
    $(LD) boot.obj sha1.obj  -release   -nologo -nodefaultlib -debug -opt:ref,icf -ltcg   -machine:AMD64 -subsystem:console,"5.02"  "D:\perl\5.32.1\AMD64\lib\CORE\perl532.lib" "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\\lib\x64\oldnames.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\kernel32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\user32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\gdi32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\winspool.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\comdlg32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\advapi32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\shell32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\ole32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\oleaut32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\netapi32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\uuid.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\ws2_32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\mpr.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\winmm.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\version.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\odbc32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\odbccp32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\comctl32.lib" "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\\lib\x64\msvcrt.lib" "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\\lib\x64\vcruntime.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\ucrt\x64\ucrt.lib"  ppresource.obj -out:boot.exe

And here's the link command that happens:

link boot.obj sha1.obj  -release   -nologo -nodefaultlib -debug -opt:ref,icf -ltcg   -machine:AMD64 -subsystem:console,"5.02"  "D:\perl\5.32.1\AMD64\lib\CORE\perl532.lib" "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\\lib\x64\oldnames.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\kernel32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\user32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\gdi32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\winspool.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\comdlg32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\advapi32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\shell32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\ole32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\oleaut32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\netapi32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\uuid.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\ws2_32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\mpr.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\winmm.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\version.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\odbc32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\odbccp32.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\um\x64\comctl32.lib" "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\\lib\x64\msvcrt.lib" "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Tools\MSVC\14.28.29333\\lib\x64\vcruntime.lib" "C:\Program Files (x86)\Windows Kits\10\lib\10.0.18362.0\ucrt\x64\ucrt.lib"  ppresource.obj -out:boot.exe

I think we'll need to look at modifying that so it's using static libs from MSVC.

rschupp commented 2 years ago

Try just

link boot.obj sha1.obj  -release   -nologo -nodefaultlib -debug -opt:ref,icf -ltcg   -machine:AMD64 -subsystem:console,"5.02"   ppresource.obj -out:boot.exe

(the equivalent line works on Linux). If not, try just adding msvcrt.lib ADVAPI32.lib KERNEL32.lib. boot.exe does not reference any perl stuff, i.e. perl532.lib is definitely not needed.

thoke commented 2 years ago

Here’s an update on what I’ve been able to accomplish…

Leveraging a “static” build of Perl, I was able to run perl Makefile.PL to generate the two PAR-Packer related Makefiles. I modified the makefiles to change the path to perl point to the dynamic build.

Then, I used the “dynamic” build of Perl to actually run nmake. At this point, all the exe files (boot.exe, par.exe, parl.exe, and parldyn.exe) all showed promising static looking results:

Dump of file myldr\boot.exe

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    KERNEL32.dll
    ADVAPI32.dll

Dump of file myldr\par.exe

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    perl532.dll
    KERNEL32.dll
    ADVAPI32.dll

Dump of file myldr\parl.exe

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    KERNEL32.dll
    ADVAPI32.dll

Dump of file myldr\parldyn.exe

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    perl532.dll
    KERNEL32.dll
    ADVAPI32.dll

To keep things simple, I just nmake installed that build into my perl. The only two files that get installed are parl.exe and parldyn.exe, but they are certainly the static looking versions as well:

Dump of file D:\perl\5.32.1\AMD64\bin\parl.exe

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    KERNEL32.dll
    ADVAPI32.dll

Dump of file D:\perl\5.32.1\AMD64\bin\parldyn.exe

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    perl532.dll
    KERNEL32.dll
    ADVAPI32.dll

I moved on to building a PAR::Packer’ed binary, but it is back to having all the dynamic pieces:

Dump of file hello-static-pp-dynamic-perl.exe

File Type: EXECUTABLE IMAGE

  Image has the following dependencies:

    KERNEL32.dll
    ADVAPI32.dll
    VCRUNTIME140.dll
    api-ms-win-crt-runtime-l1-1-0.dll
    api-ms-win-crt-string-l1-1-0.dll
    api-ms-win-crt-filesystem-l1-1-0.dll
    api-ms-win-crt-stdio-l1-1-0.dll
    api-ms-win-crt-environment-l1-1-0.dll
    api-ms-win-crt-heap-l1-1-0.dll
    api-ms-win-crt-process-l1-1-0.dll
    api-ms-win-crt-math-l1-1-0.dll
    api-ms-win-crt-locale-l1-1-0.dll

  Summary

      2F6000 .data
        1000 .pdata
        2000 .rdata
        1000 .reloc
        3000 .rsrc
        4000 .text

I ran pp with the verbose -vvv option and I see something that makes me wonder if we’re ever going to be able to make this work:

-->8 snip 8<--

D:\perl\5.32.1\AMD64\bin/pp: *** C:\Users\thoke\AppData\Local\Temp\2\ppmdbuZ.par: 625895 bytes read, 151646 compressed, 75% saved.
D:\perl\5.32.1\AMD64\bin/pp: Generating a fresh 'parl'.
D:\perl\5.32.1\AMD64\bin/pp: Running C:\Users\thoke\AppData\Local\Temp\2\parlaXZwGpu.exe -B -Ohello-static-pp-dynamic-perl.exe C:\Users\thoke\AppData\Local\Temp\2\ppmdbuZ.par

-->8 snip 8<--

That “fresh 'parl'” is probably getting all the dynamic things and then putting them into the resulting hello…exe is as well. I can’t check that because it’s a temporary thing and gets removed after the command completes.

rschupp commented 2 years ago

Dump of file myldr\boot.exe File Type: EXECUTABLE IMAGE Image has the following dependencies: KERNEL32.dll ADVAPI32.dll

Look's good

To keep things simple, I just nmake installed that build into my perl. The only two files that get installed are parl.exe and parldyn.exe ...

That doesn't look right, PAR/StrippedPARL/{Dynamic,Static}.pm should also have been installed - did you nmake install only in myldr? Or maybe the make dependencies are incomplete. Anyway, these are the files that are used to construct every pped file, so if they didn't get updated it's no surprise that your hello-static-pp-dynamic-perl.exe still references VCRUNTIME140.dll et al.

Have you tried my previous suggestion (with the non-static perl) to link boot.exe by hand without referencing any *.lib files? perl532.lib in the linker command really shouldn't be required.

thoke commented 2 years ago

I maybe missed some of the dependencies, I'll circle back and try that again to confirm. In the mean time, when I got to the point after having run nmake at the top level (against the "dynamic" Perl), then dropping into myldr and trying the manual link command, there are a bunch of unresolved symbols:

Z:\par-packer-testing\PAR-Packer\myldr>link boot.obj sha1.obj  -release   -nologo -nodefaultlib -debug -opt:ref,icf -ltcg   -machine:AMD64 -subsystem:console,"5.02"   ppresource.obj -out:boot.exe
boot.obj : error LNK2001: unresolved external symbol __security_check_cookie
boot.obj : error LNK2001: unresolved external symbol __imp_exit
boot.obj : error LNK2001: unresolved external symbol __imp_GetModuleFileNameA
boot.obj : error LNK2001: unresolved external symbol __imp_strtok
boot.obj : error LNK2001: unresolved external symbol __imp_unlink
boot.obj : error LNK2001: unresolved external symbol __imp__rmdir
boot.obj : error LNK2001: unresolved external symbol memcpy
boot.obj : error LNK2001: unresolved external symbol __report_rangecheckfailure
boot.obj : error LNK2001: unresolved external symbol __imp_strpbrk
boot.obj : error LNK2001: unresolved external symbol __imp_open
boot.obj : error LNK2001: unresolved external symbol __imp__findclose
boot.obj : error LNK2001: unresolved external symbol __imp___acrt_iob_func
boot.obj : error LNK2001: unresolved external symbol __imp_strdup
boot.obj : error LNK2001: unresolved external symbol __imp__putenv
boot.obj : error LNK2001: unresolved external symbol __imp_strnicmp
boot.obj : error LNK2001: unresolved external symbol __imp_GetModuleHandleA
boot.obj : error LNK2001: unresolved external symbol __imp_access
boot.obj : error LNK2001: unresolved external symbol __imp___stdio_common_vfprintf
boot.obj : error LNK2001: unresolved external symbol __imp__unlink
boot.obj : error LNK2001: unresolved external symbol strcmp
boot.obj : error LNK2001: unresolved external symbol __imp_strncpy
boot.obj : error LNK2001: unresolved external symbol __imp_chmod
boot.obj : error LNK2001: unresolved external symbol __imp_read
boot.obj : error LNK2001: unresolved external symbol __imp_rename
boot.obj : error LNK2001: unresolved external symbol __imp_GetUserNameA
boot.obj : error LNK2001: unresolved external symbol __imp__stat64i32
boot.obj : error LNK2001: unresolved external symbol __imp_strncmp
boot.obj : error LNK2001: unresolved external symbol __imp_lseek
boot.obj : error LNK2001: unresolved external symbol __imp_LoadLibraryA
boot.obj : error LNK2001: unresolved external symbol strlen
boot.obj : error LNK2001: unresolved external symbol __imp__errno
boot.obj : error LNK2001: unresolved external symbol __imp___stdio_common_vsprintf
boot.obj : error LNK2001: unresolved external symbol __imp__stricmp
boot.obj : error LNK2001: unresolved external symbol __imp_GetProcAddress
boot.obj : error LNK2001: unresolved external symbol __imp_free
boot.obj : error LNK2001: unresolved external symbol __imp___p__environ
boot.obj : error LNK2001: unresolved external symbol __imp__findfirst64i32
boot.obj : error LNK2001: unresolved external symbol __imp_strspn
boot.obj : error LNK2001: unresolved external symbol __imp_getpid
boot.obj : error LNK2001: unresolved external symbol __imp_FreeLibrary
boot.obj : error LNK2001: unresolved external symbol __imp_spawnvp
boot.obj : error LNK2001: unresolved external symbol strstr
boot.obj : error LNK2001: unresolved external symbol __imp__findnext64i32
boot.obj : error LNK2001: unresolved external symbol __imp__mkdir
boot.obj : error LNK2001: unresolved external symbol __imp_write
boot.obj : error LNK2001: unresolved external symbol __imp_close
boot.obj : error LNK2001: unresolved external symbol __imp_malloc
sha1.obj : error LNK2001: unresolved external symbol memset
LINK : error LNK2001: unresolved external symbol mainCRTStartup
boot.exe : fatal error LNK1120: 49 unresolved externals

I didn't really try to see if I could add anything more to the link command for now.

GlennWood commented 2 years ago

Tim has moved on up to bigger, better, and broader things in the organization, and I have taken over our Perl building responsibilities. In the nine months we have successfully rebuilt Perl and PAR multiple times on multiple platforms, including MSWin32. This problem does not recur ... except when we fumble our build process. It has to do with recompiling boot.exe (to add product-info), then trying to PP with that boot.exe. I don't think we understood that we must make sure the Dynamic.pm and Static.pm should be loaded from the PERL5LIB where the boot.exe rebuild recreated them! Too often, our PERL5LIB was mis-configured.

So this issue can be closed. Thank you for your help.

rschupp commented 2 years ago

@GlennWood Thanks for following up!