PAR issue packaging SSLeay.dll on 5.36.1 on Windows

[lmcne]

I'm seeing the following failure when attempting to pack a script using Net::SSLeay in Perl 5.36.1 on Windows:

Can't load 'C:\Users\myuser\AppData\Local\Temp\3\par-74686f6b65\temp-1772\inc\lib/auto/Net/SSLeay/SSLeay.dll' for module Net::SSLeay: load_file:The specified module could not be found at <embedded>/DynaLoader.pm line 206. at <embedded>/PAR/Heavy.pm line 95.

Switching over to 5.34.1 or earlier succeeds with no issues.

Based on testing, I think this is due to the following change made in DynaLoader (https://github.com/Perl/perl5/commit/f0e0f25d5ce9118d61837a3d286f5fd2fff7b1a3):

 20 BEGIN {
 21     our $VERSION = '1.52';
 22 }
 23
 24 # Note: in almost any other piece of code "our" would have been a better
 25 # option than "use vars", but DynaLoader's bootstrap files need the
 26 # side effect of the variable being declared in any scope whose current
 27 # package is DynaLoader, not just the current lexical one.
 28 use vars qw(@dl_library_path @dl_resolve_using @dl_require_symbols
 29             $dl_debug @dl_librefs @dl_modules @dl_shared_objects
 30             $dl_dlext $dl_so $dlsrc @args $module @dirs $file $bscode);
 31

I suspect this is affecting the following injunction in PAR::Heavy:

 18 ########################################################################
 19 # Dynamic inclusion of XS modules
 20
 21 # NOTE: Don't "use" any module here, esp. one that is an XS module or
 22 # whose "use" could cause the loading of an XS module thru its dependencies.
 23

demo scripts are available to reproduce.

[rschupp]

demo scripts are available to reproduce.

Actually someting like

pp -o foo.exe -E "use Net::SSLeay"
.\foo.exe

should be able to reproduce?

[rschupp]

Please find the file auto/Net/SSLeay/SSLeay.dll in your Perl 5.36.1 installation and run the Windows equivalent of

objdump -ax ..../auto/Net/SSLeay/SSLeay.dll   | grep "DLL Name"

on it.

[lmcne]

Here's what we see for objdump -ax on Perl 5.36.1:

SSLeay-5.36.1.out:    DLL Name: perl536.dll
SSLeay-5.36.1.out:    DLL Name: VXCRYPTO_7.dll
SSLeay-5.36.1.out:    DLL Name: VXSSL_7.dll
SSLeay-5.36.1.out:    DLL Name: KERNEL32.dll
SSLeay-5.36.1.out:    DLL Name: VCRUNTIME140.dll
SSLeay-5.36.1.out:    DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
SSLeay-5.36.1.out:    DLL Name: api-ms-win-crt-string-l1-1-0.dll
SSLeay-5.36.1.out:    DLL Name: api-ms-win-crt-time-l1-1-0.dll
SSLeay-5.36.1.out:    DLL Name: api-ms-win-crt-runtime-l1-1-0.dll

As a cross-reference, we also ran this against a working build based on 5.32.1:

SSLeay-5.32.1.out:    DLL Name: perl532.dll
SSLeay-5.32.1.out:    DLL Name: VXCRYPTO_7.dll
SSLeay-5.32.1.out:    DLL Name: VXSSL_7.dll
SSLeay-5.32.1.out:    DLL Name: KERNEL32.dll
SSLeay-5.32.1.out:    DLL Name: VCRUNTIME140.dll
SSLeay-5.32.1.out:    DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
SSLeay-5.32.1.out:    DLL Name: api-ms-win-crt-string-l1-1-0.dll
SSLeay-5.32.1.out:    DLL Name: api-ms-win-crt-time-l1-1-0.dll
SSLeay-5.32.1.out:    DLL Name: api-ms-win-crt-runtime-l1-1-0.dll

Results look identical...

As a further test we checked out Perl 5.36.1 and git reverted the last update for DynaLoader before building, effectively giving us Perl 5.36.1 with a 5.34.1 DynaLoader. This build succeeded in the same way as stock Perl 5.34.1.

The issue only appears when PAR-packing with a 5.36.x DynaLoader.

[thoke]

Here's what we see for objdump -ax on Perl 5.36.1:

SSLeay-5.36.1.out:    DLL Name: perl536.dll
SSLeay-5.36.1.out:    DLL Name: VXCRYPTO_7.dll
SSLeay-5.36.1.out:    DLL Name: VXSSL_7.dll
SSLeay-5.36.1.out:    DLL Name: KERNEL32.dll
SSLeay-5.36.1.out:    DLL Name: VCRUNTIME140.dll
SSLeay-5.36.1.out:    DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
SSLeay-5.36.1.out:    DLL Name: api-ms-win-crt-string-l1-1-0.dll
SSLeay-5.36.1.out:    DLL Name: api-ms-win-crt-time-l1-1-0.dll
SSLeay-5.36.1.out:    DLL Name: api-ms-win-crt-runtime-l1-1-0.dll

As a cross-reference, we also ran this against a working build based on 5.32.1:

SSLeay-5.32.1.out:    DLL Name: perl532.dll
SSLeay-5.32.1.out:    DLL Name: VXCRYPTO_7.dll
SSLeay-5.32.1.out:    DLL Name: VXSSL_7.dll
SSLeay-5.32.1.out:    DLL Name: KERNEL32.dll
SSLeay-5.32.1.out:    DLL Name: VCRUNTIME140.dll
SSLeay-5.32.1.out:    DLL Name: api-ms-win-crt-stdio-l1-1-0.dll
SSLeay-5.32.1.out:    DLL Name: api-ms-win-crt-string-l1-1-0.dll
SSLeay-5.32.1.out:    DLL Name: api-ms-win-crt-time-l1-1-0.dll
SSLeay-5.32.1.out:    DLL Name: api-ms-win-crt-runtime-l1-1-0.dll

Results look identical...

As a further test we checked out Perl 5.36.1 and git reverted the last update for DynaLoader before building, effectively giving us Perl 5.36.1 with a 5.34.1 DynaLoader. This build succeeded in the same way as stock Perl 5.34.1.

The issue only appears when PAR-packing with a 5.36.x DynaLoader.

Just to be fully transparent, I think the process was to:

git checkout -b release/v5.36.1 v5.36.1
git checkout v5.34.1 -- ext/DynaLoader

Then re-build Perl.

[shawnlaffan]

Perhaps just a data point, but on 5.32 the libssl, zlib and libcrypto DLLs need to be packed.

Running with pp_autolink:

pp_autolink net_ssleay.pl
DLL check iter: 1
DLL check iter: 2
DLL check iter: 3
No alien system dlls detected

Detected link list: c:\user\perls\5.32.1.1_pdl\c\bin/libssl-1_1-x64__.dll c:\user\perls\5.32.1.1_pdl\c\bin/libcrypto-1_1-x64__.dll c:\user\perls\5.32.1.1_pdl\c\bin/zlib1__.dll

Detected alien list:

CMD: pp --link c:\user\perls\5.32.1.1_pdl\c\bin/libssl-1_1-x64__.dll --link c:\user\perls\5.32.1.1_pdl\c\bin/libcrypto-1_1-x64__.dll --link c:\user\perls\5.32.1.1_pdl\c\bin/zlib1__.dll net_ssleay.pl

[lmcne]

Interesting - what exactly is net_ssleay.pl? Is that a standard test script, or something you built to prove out the behaviour?

Our build process explicitly packs libssl and libcrypto. we were packing zlib1.dll as well: Strawberry Perl ships zlib1.dll, and we were including that in our application builds until we shifted off Strawberry Perl.

When we evaluated it we couldn't find anything in our stack actually using it: possibly because we leverage Compress::Raw::Zlib, which builds its' own copy of zlib during install.

We stopped explicitly packing zlib1__.dll several months ago with no obvious impact to our executable: it behaves as expected, including HTTPS traffic.

[rschupp]

@lmcne My guess is that net_ssleay.pl is just

use Net::SSLeay.pl

that @shawnlaffan whipped up to use his pp_auto_link to show what (recursively) dependent DLLs should manually be packed (using pp --link) when a script uses Net::SSLeay. (pp_auto_link doesn't have something like pp -E) His environment is probably Strawberry Perl 5.32.1. Apparently Strawberry links either libssl or libcrypto with zlib, but that's probably not necessary, here on Linux both shared libraries are not linked with (neither for OpenSSL 1.1 nor for 3.0).

Anyway, I assume the VXSSL_7.dll and VXCRYPTO_7.dll mentioned above are your builds of libssl and libcrypto and that you have checked that they actually have been packed into your executable (using unzip -l or similar on the executable, they should be in the directory shlib/$Config{archname}) and have also checked that they have correctly been extracted into the cache folder (e.g. C:\Users\myuser\AppData\Local\Temp\3\par-74686f6b65\temp-1772 in the original issue) at the top level.

[shawnlaffan]

Yes, I should have specified that the script is just use strict; use warnings; use Net::SSLeay and run under Strawberry perl 5.32.

[lmcne]

Thanks for the clarification - I figured it was something simple, but wondered if perhaps there was a comprehensive tool out there...

Also thanks for pointing me towards pp_autolink; that looks like a really interesting tool!

[lmcne]

Can confirm: both vxcrypto_7.dll and vxssl_7.dll are present in the packed binary and are getting extracted at runtime:

 Directory of C:\Users\lmcne\AppData\Local\Temp\par-6c6d636e65726e65\cache-8f92452c315e83c35c607a8fc43e9a33bfd300e3\inc\shlib\MSWin32-x64-multi-thread

05/18/2023  02:59 PM    <DIR>          .
05/18/2023  02:59 PM    <DIR>          ..
05/18/2023  02:59 PM         2,497,328 vxcrypto_7.dll
05/18/2023  02:59 PM           372,016 vxssl_7.dll
               2 File(s)      2,869,344 bytes
               2 Dir(s)  74,236,801,024 bytes free

Contents are identical to the 5.32.1 build.

[rschupp]

OK, but they should also be present in the toplevel of the cache directory, ie. C:\Users\lmcne\AppData\Local\Temp\par-6c6d636e65726e65\cache-8f92452c315e83c35c607a8fc43e9a33bfd300e3, because that's where PATH is set when running the "inner" executable.

[lmcne]

Confirmed:

C:\Users\lmcne>dir C:\Users\lmcne\AppData\Local\Temp\par-6c6d636e65726e65\cache-8f92452c315e83c35c607a8fc43e9a33bfd300e3 | findstr /i "vx"
02/10/2023  12:10 PM         2,497,328 vxcrypto_7.dll
02/10/2023  12:10 PM           372,016 vxssl_7.dll

[rschupp]

Looks good... But why doesn't this work from the packed executable? Can you run the following (with your perl 5.36.1) and C:\Users\lmcne\AppData\Local\Temp\par-6c6d636e65726e65\cache-8f92452c315e83c35c607a8fc43e9a33bfd300e3 prepended to PATH

perl -E "use DynaLoader; my $lib = DynaLoader::dl_load_file('C:\Users\lmcne\AppData\Local\Temp\par-6c6d636e65726e65\cache-8f92452c315e83c35c607a8fc43e9a33bfd300e3\inc\lib/auto/Net/SSLeay/SSLeay.dll', 0); say $lib ? '=> '.$lib : 'error: '.DynaLoader::dl_error();"

[lmcne]

EDIT: Mysteriously the two vx*_7.dll's vanished from the cache between Friday and today. I didn't realize that when I first posted this comment, and it affected the test. I reran the executable to get a fresh cache, and the .dll's were present.

Getting this:

C:\Users\lmcne\perl\5.36.1\AMD64\5.36.1\bin\perl.exe -E "use DynaLoader; my $lib = DynaLoader::dl_load_file('C:\Users\lmcne\AppData\Local\Temp\par-6c6d636e65726e65\cache-8f92452c315e83c35c607a8fc43e9a33bfd300e3\inc\lib\auto\Net\SSLeay\SSLeay.dll', 0); say $lib ? '=> '.$lib : 'error: '.DynaLoader::dl_error();"
=> 140723660718080

[rschupp]

I just pushed a tentative fix to branch "dynaload" (should only have any effect on perl >= 5.36.0). Please try it out.

[lmcne]

Success! I pulled the dynaload branch down and packaged it into our local CPAN instance, than ran a Windows build: executable completed without errors and appears fully functional.

[rschupp]

Thanks for testing, I'll make a new release of PAR::Packer in a few days.

Note however that this is just a workaround for PAR::Packer only, https://github.com/Perl/perl5/commit/f0e0f25d5ce9118d61837a3d286f5fd2fff7b1a3 breaks loading of XS modules from .par files in general. PAR::Packer just patches out (using PAR::Filter::PatchContent) one line DynaLoader.pm from this commit: turning $do_expand from an (implicit) local variable to a my variable.

In general, PAR::Heavy monkey-patches DynaLoader at runtime so that Foo::Bar->bootstrap() will locate Foo/Bar/Bar.dll in the .par (i.e. .zip) file, extract it into $PAR_TEMP as some abcdef12_.dll, then load this DLL and call its bootstrap function.

What I still don't understand is the actual error in the original issue: the monkey-patching fails, but the correct DLL is found (since nowadays we extract the complete zip part of the packed executable into $PAR_TEMP/inc and add this to @INC), but loading it fails for unspecified reasons. This is definitely a Windows specific problem, on Linux the "wrong" DLL is loaded without problems. In https://github.com/rschupp/PAR-Packer/issues/74#issuecomment-1557410138 you checked that DynaLoader can load this DLL, but apparently not when run from the packed executable.

[lmcne]

Thank you SO MUCH for fixing this - it's greatly appreciated!