Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.99k stars 559 forks source link

[PATCH] cpan/Win32 add delay loading for GCC and VC #16570

Open p5pRT opened 6 years ago

p5pRT commented 6 years ago

Migrated from rt.perl.org#133229 (status was 'new')

Searchable as RT133229$

p5pRT commented 6 years ago

From @bulk88

Created by @bulk88

See attached patch. Followup to no responses at https://www.nntp.perl.org/group/perl.perl5.porters/2018/05/msg250977.html

Perl Info ``` --- Flags: category=core severity=low --- Site configuration information for perl 5.27.9: Configured by Administrator at Tue Jan 30 20:34:30 2018. Summary of my perl5 (revision 5 version 27 subversion 9) configuration: Platform: osname=MSWin32 osvers=5.2.3790 archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended useposix=true d_sigaction=undef useithreads=define usemultiplicity=define use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n default_inc_excludes_dot=define bincompat5005=undef Compiler: cc='cl' ccflags ='-nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32 -D_CONSOLE -DNO_STRICT -D_CRT_SECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DPERL_TEXTMODE_SCRIPTS -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DWIN32_NO_REGISTRY' optimize='-O1 -MD -Zi -DNDEBUG -GL' cppflags='-DWIN32' ccversion='15.00.30729.01' gccversion='' gccosandvers='' intsize=4 longsize=4 ptrsize=4 doublesize=8 byteorder=1234 doublekind=3 d_longlong=undef longlongsize=8 d_longdbl=define longdblsize=8 longdblkind=0 ivtype='long' ivsize=4 nvtype='double' nvsize=8 Off_t='__int64' lseeksize=8 alignbytes=8 prototype=define Linker and Libraries: ld='link' ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86' libpth="C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\lib" libs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib perllibs=oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib comctl32.lib msvcrt.lib libc=msvcrt.lib so=dll useshrplib=true libperl=perl527.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs dlext=dll d_dlsymun=undef ccdlflags=' ' cccdlflags=' ' lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -ltcg -libpath:"c:\perl\lib\CORE" -machine:x86' --- @INC for perl 5.27.9: lib C:/p527/srcnew/lib --- Environment for perl 5.27.9: CYGWIN=tty HOME (unset) LANG (unset) LANGUAGE (unset) LD_LIBRARY_PATH=/usr/lib/x86:/usr/X11R6/lib LOGDIR (unset) PATH=C:\WINDOWS\system32;C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\BIN;C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin;C:\Perl\bin;C:\WINDOWS;C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE;C:\Program Files (x86)\Git\bin;C:\sp3220\c\bin; PERL_BADLANG (unset) SHELL (unset) ```
p5pRT commented 6 years ago

From @bulk88

0001-cpan-Win32-add-delay-loading-for-GCC-and-VC.patch ```diff From 1e690bc6fdd1f7eb3cf49ee9fdbec6443a6dcc85 Mon Sep 17 00:00:00 2001 From: Daniel Dragan Date: Thu, 17 May 2018 01:35:27 -0400 Subject: [PATCH] cpan/Win32 add delay loading for GCC and VC Most common use of Win32:: is cwd() and short vs long or rel vs abs path functions. Followed by GetLastError. version.dll is very rarely used. ole32.dll loads RPC service registration and COM and registry stuff into the process. To speed up all perl modules EUMM building and later testing. Put these 2 DLLs to be delay loaded. Visual C is easy to delay load with. With GCC it is complicated, with little to no known public use of the feature, and the feature is crudely hacked into LD without LD being aware of what delay loaded DLLs are. Technically, the linker and OS PE loader never need to be aware what delay loaded DLLs are. Its just a pointer table initially all pointing at one var arg, CPU context saving (think setjmp) function written in asm, that loads the DLL, and writes the func ptr into the array, then longjmp()s into the newly fetched function in another DLL. Subsequent calls goto the real function and not the vararg loader one. There is one serious bug with GCC delay loading, described at https://sourceware.org/bugzilla/show_bug.cgi?id=14339 . I have worked around that. And a spec violation that the delay loading structs arent mentioned in the PE header. So static PE analysis tools dont show the delayed imports for GCC, unlike for VC binaries where delayed imports are listed by public PE tools. Delayed imports not being mentioned in the PE header doesn't affect the delayed feature. But it prevents "binding" the delayed imports to OS DLLs (perl doesn't currently do this, maybe one day it will if I publish the code) and it prevents debugging tools from seeing them. To fix this second spec violation, compute the RVA of the delayed import struct array and write it into the PE header after the linker puts the DLL on disk. Use a GCC map file to find the abs addr of the struct inside the DLL. Also add a null termination array slice "nulldlydescr.c" to stop debugging tools from crashing or throwing errors, since VC always generates the null filled struct. I named the symbol _NULL_DELAY_IMPORT_DESCRIPTOR after how VC internally names it, but VC uses a different pattern of symbol names than dlltool does. dlltool is "_DELAY_IMPORT_DESCRIPTOR_ole32_dlya" VC is "__DELAY_IMPORT_DESCRIPTOR_ole32"/"__NULL_DELAY_IMPORT_DESCRIPTOR" --- cpan/Win32/.gitignore | 4 ++ cpan/Win32/Makefile.PL | 109 ++++++++++++++++++++++++++++++++ cpan/Win32/Win32.xs | 34 ++++++++++ cpan/Win32/mkdlylib.PL | 49 +++++++++++++++ cpan/Win32/nulldlydescr.c | 13 ++++ cpan/Win32/setdlyhdr.PL | 157 ++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 366 insertions(+) create mode 100644 cpan/Win32/.gitignore create mode 100644 cpan/Win32/mkdlylib.PL create mode 100644 cpan/Win32/nulldlydescr.c create mode 100644 cpan/Win32/setdlyhdr.PL diff --git a/cpan/Win32/.gitignore b/cpan/Win32/.gitignore new file mode 100644 index 0000000..c16cbf7 --- /dev/null +++ b/cpan/Win32/.gitignore @@ -0,0 +1,4 @@ +*.tdef +*.dlya +*.s +*.map diff --git a/cpan/Win32/Makefile.PL b/cpan/Win32/Makefile.PL index 0f16594..2ab68ee 100644 --- a/cpan/Win32/Makefile.PL +++ b/cpan/Win32/Makefile.PL @@ -2,17 +2,126 @@ use 5.006; use strict; use warnings; use ExtUtils::MakeMaker; +use Config; unless ($^O eq "MSWin32" || $^O eq "cygwin") { die "OS unsupported\n"; } +#use delay loading for version.dll and ole32.dll +#if true, checks will be performed if delay loading can be done +#if checks fail, delay loading will not be done +my $CanMSVCDelayLoad = 1; +my $CanGCCDelayLoad = 1; + +#set this macro here, additional stuff might be appended +my $OTHERLDFLAGS = ''; + +CheckMSVCDelayLoad(); +CheckGCCDelayLoad(); my %param = ( NAME => 'Win32', VERSION_FROM => 'Win32.pm', INSTALLDIRS => ($] >= 5.008004 && $] < 5.012 ? 'perl' : 'site'), + PL_FILES => {}, + DEFINE => ($CanGCCDelayLoad ? '-DMY_GCC_DELAY_LOAD' : ''), + clean => {FILES => '*.tdef *.dlya *.s *.map'}, + dynamic_lib => { + OTHERLDFLAGS => + $OTHERLDFLAGS . ($CanMSVCDelayLoad ? + ' -DELAYLOAD:version.dll -DELAYLOAD:ole32.dll delayimp.lib ' + : $CanGCCDelayLoad ? ' -Wl,-Map=$(BASEEXT).map ' + : '') + } ); $param{NO_META} = 1 if eval "$ExtUtils::MakeMaker::VERSION" >= 6.10_03; $param{LIBS} = ['-L/lib/w32api -lole32 -lversion'] if $^O eq "cygwin"; WriteMakefile(%param); + +sub CheckMSVCDelayLoad { + my $LocalCanMSVCDelayLoad = 0; + if($CanMSVCDelayLoad && $Config{ld} eq 'link') { + my $linkoutput = `link /?`; + if($linkoutput =~ /delayload/i) { + $LocalCanMSVCDelayLoad = 1; + } + } + $CanMSVCDelayLoad = $LocalCanMSVCDelayLoad; + print "CanMSVCDelayLoad= $CanMSVCDelayLoad\n"; +} + +sub CheckGCCDelayLoad{ + my $LocalCanGCCDelayLoad = 0; #cygwin not tested so dly ld disabled + if($CanGCCDelayLoad && $Config{cc} eq 'gcc' && $^O ne "cygwin") { + my $dlltoolhelp = `dlltool -h`; + my $nm = `nm -V`; + if(index($dlltoolhelp , 'output-delaylib') != -1 + && index($nm, 'GNU General Public License') != -1) { + my $libpaths; + my $cmd = "$Config{cc} -print-search-dirs"; + foreach(`$cmd`) { + if($_ =~ /^libraries: =(.+)$/) { + $libpaths = $1; + last; + } + } + die "command \"$cmd\" failed to return a library search path" + unless $libpaths; + my @libpaths = split(';', $libpaths); +# add compiler default lib paths (not from %Config or env) to EU::LL's psuedo +# lib search dirs since most GCC Win32 builds have CCHOME AKA $Config{libpth} +# set wrong + my @mingwexarr = ExtUtils::Liblist->ext( + join(' ', map(qq["-L$_"], @libpaths)).' -lmingwex :nodefault', + 1, 1 + ); + if($mingwexarr[0] =~ /mingwex/i){ + my $runstr = 'nm -g "'.$mingwexarr[0].'"'; + my $mingwexdump = `$runstr`; + if(index($mingwexdump,'__delayLoadHelper2') != -1){ + $LocalCanGCCDelayLoad = 1; + } + } + } + } + $CanGCCDelayLoad = $LocalCanGCCDelayLoad; + print "CanGCCDelayLoad= $CanGCCDelayLoad\n"; +} + +package MY; + +sub postamble { + return +#note %.dlya is dmake only syntax +($CanGCCDelayLoad ? ' +$(INST_DYNAMIC) : version.dlya ole32.dlya nulldlydescr.o + +%.dlya : mkdlylib.PL + $(PERLRUN) mkdlylib.PL '.$main::Config{cc}.' $* + +':''); + +} + +sub const_loadlibs { + my ($self) = @_; + if($CanGCCDelayLoad) { + #ordering of nulldlydescr.o is very important, must be after all .dlya files + $self->{LDLOADLIBS} .= ' version.dlya ole32.dlya nulldlydescr.o'; + $self->{LDLOADLIBS} =~ s/-lversion//; + $self->{LDLOADLIBS} =~ s/-lole32//; + } + return $self->SUPER::const_loadlibs; +} + +sub dynamic_lib { + my $self = shift; #note OTHERLDFLAGS is in @_ + my $block = $self->SUPER::dynamic_lib(@_); + if($CanGCCDelayLoad) { + substr($block, -1, 1, ' + $(PERLRUN) setdlyhdr.PL $@ $(BASEEXT).map +'); + } + return $block; +} diff --git a/cpan/Win32/Win32.xs b/cpan/Win32/Win32.xs index de3764e..69abdcf 100644 --- a/cpan/Win32/Win32.xs +++ b/cpan/Win32/Win32.xs @@ -12,6 +12,40 @@ # define countof(array) (sizeof (array) / sizeof (*(array))) #endif +#ifdef MY_GCC_DELAY_LOAD +# include +/*attribute dllimport causes access vio to integer 6 crash, the _imp_ is corrupt +in the delay load libs dlltool generated, the mingw headers use attribute +dllimport, get rid of attribute dllimport and the bug goes away + +see https://sourceware.org/bugzilla/show_bug.cgi?id=14339 +*/ +# define GCC_VERSION (__GNUC__ * 10000 \ + + __GNUC_MINOR__ * 100 \ + + __GNUC_PATCHLEVEL__) +/*pop push added in gcc 4.6*/ +# if GCC_VERSION >= 40600 +# pragma GCC diagnostic push +# endif +# pragma GCC diagnostic ignored "-Wattributes" +extern HRESULT WINAPI CoCreateGuid(GUID * pguid); +extern HRESULT WINAPI StringFromCLSID(const IID * const rclsid, LPOLESTR * lplpsz); +extern void WINAPI CoTaskMemFree(LPVOID pv); +# if GCC_VERSION >= 40600 +# pragma GCC diagnostic pop +# else +# pragma GCC diagnostic warning "-Wattributes" +# endif +/* test offsets in setdlyhdr.pl */ +# ifdef WIN64 +/* prob wrong */ +STATIC_ASSERT_DECL(STRUCT_OFFSET(IMAGE_NT_HEADERS64,OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT])==0xE0); +# else +STATIC_ASSERT_DECL(STRUCT_OFFSET(IMAGE_NT_HEADERS32,OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT])==0xE0); +# endif +STATIC_ASSERT_DECL(sizeof(ImgDelayDescr) == 0x20); +#endif + #define SE_SHUTDOWN_NAMEA "SeShutdownPrivilege" #ifndef WC_NO_BEST_FIT_CHARS diff --git a/cpan/Win32/mkdlylib.PL b/cpan/Win32/mkdlylib.PL new file mode 100644 index 0000000..5953325 --- /dev/null +++ b/cpan/Win32/mkdlylib.PL @@ -0,0 +1,49 @@ +#!/usr/bin/perl -w +use strict; +require ExtUtils::Liblist; +#file extension meanings, selected for easy makefile cleaning +#tdef = temp def file +#dlya = delay a file + +my $cc = $ARGV[0]; +my $basename = $ARGV[1]; +die "usage: perl mkdlylib.PL cc libbasename" unless $cc && $basename; +my $dotaname = 'lib'.$basename.'.a'; + +my $libpaths; +my $cmd = "$cc -print-search-dirs"; +foreach(`$cmd`) { + if($_ =~ /^libraries: =(.+)$/) { + $libpaths = $1; + last; + } +} +die "command \"$cmd\" failed to return a library search path" unless $libpaths; +my @libpaths = split(';', $libpaths); +my @arr = ExtUtils::Liblist->ext( +# add compiler default lib paths (not from %Config or env) to EU::LL's psuedo +# lib search dirs since most GCC Win32 builds have CCHOME AKA $Config{libpth} +# set wrong + join(' ', map(qq["-L$_"], @libpaths)).' -l'.$basename.' :nodefault' + , 0, 1 +); +my $libpath; +#remove the search loop, just 1 elem now +foreach(@{$arr[4]}) { + if(index($_, $dotaname) != -1){ + $libpath = $_; + last; + } +} +die("ExtUtils::Liblist can't find $dotaname") if ! defined($libpath); +my $nmout =`nm -g $libpath`; +my @exports = $nmout =~ m/^[[:xdigit:]]+ T _*(.+)$/mg; +die "nm failed, output of nm was:\n\n".$nmout if !@exports; +my $defstr = 'LIBRARY '.$basename.'.dll +EXPORTS +'.join("\n", @exports)."\n"; +open(FILE, '>', $basename.'.tdef') or die $!; +print(FILE $defstr) or die $!; +close(FILE) or die $!; +my $retcode = system('dlltool -y '.$basename.'.dlya --input-def '.$basename.'.tdef'); +die "dlltool failed with exit code ".($retcode >> 8) if $retcode != 0; diff --git a/cpan/Win32/nulldlydescr.c b/cpan/Win32/nulldlydescr.c new file mode 100644 index 0000000..6264e82 --- /dev/null +++ b/cpan/Win32/nulldlydescr.c @@ -0,0 +1,13 @@ +/* Dont bother with headers, just make a zero filled ImgDelayDescr struct. + * This file is only used with GCC delay loading, to correct the array of + * ImgDelayDescr struct to meet Visual C linker/PE debugging tool standards + */ +char _NULL_DELAY_IMPORT_DESCRIPTOR [0x20] + __attribute__ ((section (".text$2"))) +/* GCC by default will create the object file section with 32 byte alignment + probably because this object is 32 bytes long but all ImgDelayDescr structs + created by dlltool have 4 byte align, and we dont want the linker to put + padding (aka uninit data) between ImgDelayDescr structs +*/ + __attribute__ ((aligned (4))) + = {}; diff --git a/cpan/Win32/setdlyhdr.PL b/cpan/Win32/setdlyhdr.PL new file mode 100644 index 0000000..5d0dca8 --- /dev/null +++ b/cpan/Win32/setdlyhdr.PL @@ -0,0 +1,157 @@ +#!perl -w +use strict; +use Data::Dumper; +#derived from Perl core win32/bin/exetype.pl + +# All the IMAGE_* structures are defined in the WINNT.H file +# of the Microsoft Platform SDK. + +unless (0 < @ARGV && @ARGV < 3) { + print "Usage: $0 dllexefile mapfile\n"; + exit 1; +} + +my ($record,$magic,$signature,$offset,$va, $size, $win64); +open EXE, '+<', $ARGV[0] or die "Cannot open $ARGV[0]: $!\n"; +binmode EXE; + +# read IMAGE_DOS_HEADER structure +read EXE, $record, 64; +($magic,$offset) = unpack "Sx58L", $record; + +die "$ARGV[0] is not an MSDOS executable file.\n" + unless $magic == 0x5a4d; # "MZ" + +# read signature, IMAGE_FILE_HEADER and first WORD of IMAGE_OPTIONAL_HEADER +seek EXE, $offset, 0; +read EXE, $record, 4+20+2; +($signature,$size,$magic) = unpack "Lx16Sx2S", $record; + +die "PE header not found" unless $signature == 0x4550; # "PE\0\0" + +if($size == 224 && $magic == 0x10b) { # IMAGE_NT_OPTIONAL_HDR32_MAGIC + $win64 = 0; +} elsif ($size == 240 && $magic == 0x20b) { # IMAGE_NT_OPTIONAL_HDR64_MAGIC + $win64 = 1; +} else { + die "Optional header is neither in NT32 nor in NT64 format"; +} + +# Offset 0xE0 in the IMAGE_OPTIONAL_HEADER(32|64) is +# OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT] +seek EXE, $offset+0xE0, 0; +read EXE, $record, 8; +($va,$size) = unpack "LL", $record; +if ($va == 0 && $size == 0) { + open MAP, '<', $ARGV[1] or die "Cannot open $ARGV[1]: $!\n"; + binmode MAP; + { + $/ = undef; + my $map = ; + #baseaddr could also be extracted from PE header + $map =~ /__image_base__ = (0x[0-9a-f]+)/; + my $baseaddr = hex($1); + die "base address of PE file not found" unless $baseaddr; + my @descriptors = $map =~ + / (0x[0-9a-f]+) (?:_DELAY_IMPORT_DESCRIPTOR_|_NULL_DELAY_IMPORT_DESCRIPTOR)/g; + die "no delay loaded libraries found in $ARGV[1]" unless(@descriptors); + #TODO make sure all descriptors are 0x20 apart incase linker's design + #changes in future + $va = hex($descriptors[0])-$baseaddr; + $size = @descriptors*0x20 + } + printf("adding IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT VA=0x%x Size=0x%x", $va, $size); + seek EXE, $offset+0xE0, 0; + print EXE pack "LL", $va, $size; +} else { + die "Found existing delayed import header entry, not continuing"; +} +close EXE; +__END__ + +Visual C crashes reading a dll without the NULL thunk. Theoretically Visual C +should be using +OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT].Size +to iterate +over the array of ImgDelayDescr structs but instead it uses "null termination". + +Other 3rd party PE tools like PE Explorer and Ida throw errors trying to read +the x86 machine code as RVAs/pointers after these ImgDelayDescr structs, so null +termination instead of .Size member seems to the universal implementation for +parsing delay import descriptors. The ImgDelayDescr structs are allocated in +.text section because of dlltool's/ld's implementation (or lack thereof) of +delay loading which explains why there is machine code after the structs. + +C:\perl\src\win32> dumpbin /imports ..\lib\auto\win32\Win32.dll +Microsoft (R) COFF/PE Dumper Version 7.10.6030 +Copyright (C) Microsoft Corporation. All rights reserved. + + +Dump of file ..\lib\auto\win32\Win32.dll + +File Type: DLL + + Section contains the following delay load imports: + + version.dll + 00000001 Characteristics + 70A49008 Address of HMODULE + 70A50514 Import Address Table + 70A502C8 Import Name Table + 00000000 Bound Import Name Table + 00000000 Unload Import Name Table + 0 time date stamp + + 70A47586 0 GetFileVersionInfoA + 70A47576 1 GetFileVersionInfoSizeA + 70A47566 A VerQueryValueA + + ole32.dll + 00000001 Characteristics + 70A4900C Address of HMODULE + 70A50500 Import Address Table + 70A502B4 Import Name Table + 00000000 Bound Import Name Table + 00000000 Unload Import Name Table + 0 time date stamp + + 70A475CA F CoCreateGuid + 70A475BA 6A CoTaskMemFree + 70A475AA 13E StringFromCLSID + + (null) + A11CEC83 Characteristics + D0C804C7 Address of HMODULE + FA14A4A0 Import Address Table + 16FA82444 Import Name Table + 115A6E815 Bound Import Name Table + 101349070 Unload Import Name Table + 90669066 time date stamp + + +DUMPBIN : fatal error LNK1000: Internal error during DumpDelayLoadImports + + Version 7.10.6030 + + ExceptionCode = C0000005 + ExceptionFlags = 00000000 + ExceptionAddress = 0043B42A (00400000) "C:\Program Files\Microsoft Vis +ual Studio .NET 2003\Vc7\bin\link.exe" + NumberParameters = 00000002 + ExceptionInformation[ 0] = 00000000 + ExceptionInformation[ 1] = 7EFDE044 + +CONTEXT: + Eax = FF03E044 Esp = 0012E6E0 + Ebx = FF03E044 Ebp = 0012E7BC + Ecx = 0000DC00 Esi = 011A5AC8 + Edx = 7FFA0000 Edi = 00000000 + Eip = 0043B42A EFlags = 00010286 + SegCs = 0000001B SegDs = 00000023 + SegSs = 00000023 SegEs = 00000023 + SegFs = 0000003B SegGs = 00000000 + Dr0 = 0012E6E0 Dr3 = FF03E044 + Dr1 = 0012E7BC Dr6 = 0000DC00 + Dr2 = 00000000 Dr7 = 00000000 + +C:\perl\src\win32> -- 2.5.0.windows.1 ```
p5pRT commented 6 years ago

From @bulk88

Version bumped patch\, but this patch is a working POC. My main goal is getting winsock in libperl delay loaded with GCC.

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 6 years ago

From @bulk88

0001-cpan-Win32-add-delay-loading-for-GCC-and-VC.patch ```diff From 5731cb898c2b88de54c47b5cb994bd78fa41dbed Mon Sep 17 00:00:00 2001 From: Daniel Dragan Date: Thu, 31 May 2018 02:14:07 -0400 Subject: [PATCH] cpan/Win32 add delay loading for GCC and VC Most common use of Win32:: is cwd() and short vs long or rel vs abs path functions. Followed by GetLastError. version.dll is very rarely used. ole32.dll loads RPC service registration and COM and registry stuff into the process. To speed up all perl modules EUMM building and later testing. Put these 2 DLLs to be delay loaded. Visual C is easy to delay load with. With GCC it is complicated, with little to no known public use of the feature, and the feature is crudely hacked into LD without LD being aware of what delay loaded DLLs are. Technically, the linker and OS PE loader never need to be aware what delay loaded DLLs are. Its just a pointer table initially all pointing at one var arg, CPU context saving (think setjmp) function written in asm, that loads the DLL, and writes the func ptr into the array, then longjmp()s into the newly fetched function in another DLL. Subsequent calls goto the real function and not the vararg loader one. There is one serious bug with GCC delay loading, described at https://sourceware.org/bugzilla/show_bug.cgi?id=14339 . I have worked around that. And a spec violation that the delay loading structs arent mentioned in the PE header. So static PE analysis tools dont show the delayed imports for GCC, unlike for VC binaries where delayed imports are listed by public PE tools. Delayed imports not being mentioned in the PE header doesn't affect the delayed feature. But it prevents "binding" the delayed imports to OS DLLs (perl doesn't currently do this, maybe one day it will if I publish the code) and it prevents debugging tools from seeing them. To fix this second spec violation, compute the RVA of the delayed import struct array and write it into the PE header after the linker puts the DLL on disk. Use a GCC map file to find the abs addr of the struct inside the DLL. Also add a null termination array slice "nulldlydescr.c" to stop debugging tools from crashing or throwing errors, since VC always generates the null filled struct. I named the symbol _NULL_DELAY_IMPORT_DESCRIPTOR after how VC internally names it, but VC uses a different pattern of symbol names than dlltool does. dlltool is "_DELAY_IMPORT_DESCRIPTOR_ole32_dlya" VC is "__DELAY_IMPORT_DESCRIPTOR_ole32"/"__NULL_DELAY_IMPORT_DESCRIPTOR" --- cpan/Win32/.gitignore | 4 ++ cpan/Win32/Makefile.PL | 109 ++++++++++++++++++++++++++++++++ cpan/Win32/Win32.pm | 2 +- cpan/Win32/Win32.xs | 34 ++++++++++ cpan/Win32/mkdlylib.PL | 49 +++++++++++++++ cpan/Win32/nulldlydescr.c | 13 ++++ cpan/Win32/setdlyhdr.PL | 157 ++++++++++++++++++++++++++++++++++++++++++++++ 7 files changed, 367 insertions(+), 1 deletion(-) create mode 100644 cpan/Win32/.gitignore create mode 100644 cpan/Win32/mkdlylib.PL create mode 100644 cpan/Win32/nulldlydescr.c create mode 100644 cpan/Win32/setdlyhdr.PL diff --git a/cpan/Win32/.gitignore b/cpan/Win32/.gitignore new file mode 100644 index 0000000..c16cbf7 --- /dev/null +++ b/cpan/Win32/.gitignore @@ -0,0 +1,4 @@ +*.tdef +*.dlya +*.s +*.map diff --git a/cpan/Win32/Makefile.PL b/cpan/Win32/Makefile.PL index 0f16594..2ab68ee 100644 --- a/cpan/Win32/Makefile.PL +++ b/cpan/Win32/Makefile.PL @@ -2,17 +2,126 @@ use 5.006; use strict; use warnings; use ExtUtils::MakeMaker; +use Config; unless ($^O eq "MSWin32" || $^O eq "cygwin") { die "OS unsupported\n"; } +#use delay loading for version.dll and ole32.dll +#if true, checks will be performed if delay loading can be done +#if checks fail, delay loading will not be done +my $CanMSVCDelayLoad = 1; +my $CanGCCDelayLoad = 1; + +#set this macro here, additional stuff might be appended +my $OTHERLDFLAGS = ''; + +CheckMSVCDelayLoad(); +CheckGCCDelayLoad(); my %param = ( NAME => 'Win32', VERSION_FROM => 'Win32.pm', INSTALLDIRS => ($] >= 5.008004 && $] < 5.012 ? 'perl' : 'site'), + PL_FILES => {}, + DEFINE => ($CanGCCDelayLoad ? '-DMY_GCC_DELAY_LOAD' : ''), + clean => {FILES => '*.tdef *.dlya *.s *.map'}, + dynamic_lib => { + OTHERLDFLAGS => + $OTHERLDFLAGS . ($CanMSVCDelayLoad ? + ' -DELAYLOAD:version.dll -DELAYLOAD:ole32.dll delayimp.lib ' + : $CanGCCDelayLoad ? ' -Wl,-Map=$(BASEEXT).map ' + : '') + } ); $param{NO_META} = 1 if eval "$ExtUtils::MakeMaker::VERSION" >= 6.10_03; $param{LIBS} = ['-L/lib/w32api -lole32 -lversion'] if $^O eq "cygwin"; WriteMakefile(%param); + +sub CheckMSVCDelayLoad { + my $LocalCanMSVCDelayLoad = 0; + if($CanMSVCDelayLoad && $Config{ld} eq 'link') { + my $linkoutput = `link /?`; + if($linkoutput =~ /delayload/i) { + $LocalCanMSVCDelayLoad = 1; + } + } + $CanMSVCDelayLoad = $LocalCanMSVCDelayLoad; + print "CanMSVCDelayLoad= $CanMSVCDelayLoad\n"; +} + +sub CheckGCCDelayLoad{ + my $LocalCanGCCDelayLoad = 0; #cygwin not tested so dly ld disabled + if($CanGCCDelayLoad && $Config{cc} eq 'gcc' && $^O ne "cygwin") { + my $dlltoolhelp = `dlltool -h`; + my $nm = `nm -V`; + if(index($dlltoolhelp , 'output-delaylib') != -1 + && index($nm, 'GNU General Public License') != -1) { + my $libpaths; + my $cmd = "$Config{cc} -print-search-dirs"; + foreach(`$cmd`) { + if($_ =~ /^libraries: =(.+)$/) { + $libpaths = $1; + last; + } + } + die "command \"$cmd\" failed to return a library search path" + unless $libpaths; + my @libpaths = split(';', $libpaths); +# add compiler default lib paths (not from %Config or env) to EU::LL's psuedo +# lib search dirs since most GCC Win32 builds have CCHOME AKA $Config{libpth} +# set wrong + my @mingwexarr = ExtUtils::Liblist->ext( + join(' ', map(qq["-L$_"], @libpaths)).' -lmingwex :nodefault', + 1, 1 + ); + if($mingwexarr[0] =~ /mingwex/i){ + my $runstr = 'nm -g "'.$mingwexarr[0].'"'; + my $mingwexdump = `$runstr`; + if(index($mingwexdump,'__delayLoadHelper2') != -1){ + $LocalCanGCCDelayLoad = 1; + } + } + } + } + $CanGCCDelayLoad = $LocalCanGCCDelayLoad; + print "CanGCCDelayLoad= $CanGCCDelayLoad\n"; +} + +package MY; + +sub postamble { + return +#note %.dlya is dmake only syntax +($CanGCCDelayLoad ? ' +$(INST_DYNAMIC) : version.dlya ole32.dlya nulldlydescr.o + +%.dlya : mkdlylib.PL + $(PERLRUN) mkdlylib.PL '.$main::Config{cc}.' $* + +':''); + +} + +sub const_loadlibs { + my ($self) = @_; + if($CanGCCDelayLoad) { + #ordering of nulldlydescr.o is very important, must be after all .dlya files + $self->{LDLOADLIBS} .= ' version.dlya ole32.dlya nulldlydescr.o'; + $self->{LDLOADLIBS} =~ s/-lversion//; + $self->{LDLOADLIBS} =~ s/-lole32//; + } + return $self->SUPER::const_loadlibs; +} + +sub dynamic_lib { + my $self = shift; #note OTHERLDFLAGS is in @_ + my $block = $self->SUPER::dynamic_lib(@_); + if($CanGCCDelayLoad) { + substr($block, -1, 1, ' + $(PERLRUN) setdlyhdr.PL $@ $(BASEEXT).map +'); + } + return $block; +} diff --git a/cpan/Win32/Win32.pm b/cpan/Win32/Win32.pm index 7b9ab45..a9126f8 100644 --- a/cpan/Win32/Win32.pm +++ b/cpan/Win32/Win32.pm @@ -8,7 +8,7 @@ package Win32; require DynaLoader; @ISA = qw|Exporter DynaLoader|; - $VERSION = '0.52'; + $VERSION = '0.52_01'; $XS_VERSION = $VERSION; $VERSION = eval $VERSION; diff --git a/cpan/Win32/Win32.xs b/cpan/Win32/Win32.xs index de3764e..69abdcf 100644 --- a/cpan/Win32/Win32.xs +++ b/cpan/Win32/Win32.xs @@ -12,6 +12,40 @@ # define countof(array) (sizeof (array) / sizeof (*(array))) #endif +#ifdef MY_GCC_DELAY_LOAD +# include +/*attribute dllimport causes access vio to integer 6 crash, the _imp_ is corrupt +in the delay load libs dlltool generated, the mingw headers use attribute +dllimport, get rid of attribute dllimport and the bug goes away + +see https://sourceware.org/bugzilla/show_bug.cgi?id=14339 +*/ +# define GCC_VERSION (__GNUC__ * 10000 \ + + __GNUC_MINOR__ * 100 \ + + __GNUC_PATCHLEVEL__) +/*pop push added in gcc 4.6*/ +# if GCC_VERSION >= 40600 +# pragma GCC diagnostic push +# endif +# pragma GCC diagnostic ignored "-Wattributes" +extern HRESULT WINAPI CoCreateGuid(GUID * pguid); +extern HRESULT WINAPI StringFromCLSID(const IID * const rclsid, LPOLESTR * lplpsz); +extern void WINAPI CoTaskMemFree(LPVOID pv); +# if GCC_VERSION >= 40600 +# pragma GCC diagnostic pop +# else +# pragma GCC diagnostic warning "-Wattributes" +# endif +/* test offsets in setdlyhdr.pl */ +# ifdef WIN64 +/* prob wrong */ +STATIC_ASSERT_DECL(STRUCT_OFFSET(IMAGE_NT_HEADERS64,OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT])==0xE0); +# else +STATIC_ASSERT_DECL(STRUCT_OFFSET(IMAGE_NT_HEADERS32,OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT])==0xE0); +# endif +STATIC_ASSERT_DECL(sizeof(ImgDelayDescr) == 0x20); +#endif + #define SE_SHUTDOWN_NAMEA "SeShutdownPrivilege" #ifndef WC_NO_BEST_FIT_CHARS diff --git a/cpan/Win32/mkdlylib.PL b/cpan/Win32/mkdlylib.PL new file mode 100644 index 0000000..5953325 --- /dev/null +++ b/cpan/Win32/mkdlylib.PL @@ -0,0 +1,49 @@ +#!/usr/bin/perl -w +use strict; +require ExtUtils::Liblist; +#file extension meanings, selected for easy makefile cleaning +#tdef = temp def file +#dlya = delay a file + +my $cc = $ARGV[0]; +my $basename = $ARGV[1]; +die "usage: perl mkdlylib.PL cc libbasename" unless $cc && $basename; +my $dotaname = 'lib'.$basename.'.a'; + +my $libpaths; +my $cmd = "$cc -print-search-dirs"; +foreach(`$cmd`) { + if($_ =~ /^libraries: =(.+)$/) { + $libpaths = $1; + last; + } +} +die "command \"$cmd\" failed to return a library search path" unless $libpaths; +my @libpaths = split(';', $libpaths); +my @arr = ExtUtils::Liblist->ext( +# add compiler default lib paths (not from %Config or env) to EU::LL's psuedo +# lib search dirs since most GCC Win32 builds have CCHOME AKA $Config{libpth} +# set wrong + join(' ', map(qq["-L$_"], @libpaths)).' -l'.$basename.' :nodefault' + , 0, 1 +); +my $libpath; +#remove the search loop, just 1 elem now +foreach(@{$arr[4]}) { + if(index($_, $dotaname) != -1){ + $libpath = $_; + last; + } +} +die("ExtUtils::Liblist can't find $dotaname") if ! defined($libpath); +my $nmout =`nm -g $libpath`; +my @exports = $nmout =~ m/^[[:xdigit:]]+ T _*(.+)$/mg; +die "nm failed, output of nm was:\n\n".$nmout if !@exports; +my $defstr = 'LIBRARY '.$basename.'.dll +EXPORTS +'.join("\n", @exports)."\n"; +open(FILE, '>', $basename.'.tdef') or die $!; +print(FILE $defstr) or die $!; +close(FILE) or die $!; +my $retcode = system('dlltool -y '.$basename.'.dlya --input-def '.$basename.'.tdef'); +die "dlltool failed with exit code ".($retcode >> 8) if $retcode != 0; diff --git a/cpan/Win32/nulldlydescr.c b/cpan/Win32/nulldlydescr.c new file mode 100644 index 0000000..6264e82 --- /dev/null +++ b/cpan/Win32/nulldlydescr.c @@ -0,0 +1,13 @@ +/* Dont bother with headers, just make a zero filled ImgDelayDescr struct. + * This file is only used with GCC delay loading, to correct the array of + * ImgDelayDescr struct to meet Visual C linker/PE debugging tool standards + */ +char _NULL_DELAY_IMPORT_DESCRIPTOR [0x20] + __attribute__ ((section (".text$2"))) +/* GCC by default will create the object file section with 32 byte alignment + probably because this object is 32 bytes long but all ImgDelayDescr structs + created by dlltool have 4 byte align, and we dont want the linker to put + padding (aka uninit data) between ImgDelayDescr structs +*/ + __attribute__ ((aligned (4))) + = {}; diff --git a/cpan/Win32/setdlyhdr.PL b/cpan/Win32/setdlyhdr.PL new file mode 100644 index 0000000..5d0dca8 --- /dev/null +++ b/cpan/Win32/setdlyhdr.PL @@ -0,0 +1,157 @@ +#!perl -w +use strict; +use Data::Dumper; +#derived from Perl core win32/bin/exetype.pl + +# All the IMAGE_* structures are defined in the WINNT.H file +# of the Microsoft Platform SDK. + +unless (0 < @ARGV && @ARGV < 3) { + print "Usage: $0 dllexefile mapfile\n"; + exit 1; +} + +my ($record,$magic,$signature,$offset,$va, $size, $win64); +open EXE, '+<', $ARGV[0] or die "Cannot open $ARGV[0]: $!\n"; +binmode EXE; + +# read IMAGE_DOS_HEADER structure +read EXE, $record, 64; +($magic,$offset) = unpack "Sx58L", $record; + +die "$ARGV[0] is not an MSDOS executable file.\n" + unless $magic == 0x5a4d; # "MZ" + +# read signature, IMAGE_FILE_HEADER and first WORD of IMAGE_OPTIONAL_HEADER +seek EXE, $offset, 0; +read EXE, $record, 4+20+2; +($signature,$size,$magic) = unpack "Lx16Sx2S", $record; + +die "PE header not found" unless $signature == 0x4550; # "PE\0\0" + +if($size == 224 && $magic == 0x10b) { # IMAGE_NT_OPTIONAL_HDR32_MAGIC + $win64 = 0; +} elsif ($size == 240 && $magic == 0x20b) { # IMAGE_NT_OPTIONAL_HDR64_MAGIC + $win64 = 1; +} else { + die "Optional header is neither in NT32 nor in NT64 format"; +} + +# Offset 0xE0 in the IMAGE_OPTIONAL_HEADER(32|64) is +# OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT] +seek EXE, $offset+0xE0, 0; +read EXE, $record, 8; +($va,$size) = unpack "LL", $record; +if ($va == 0 && $size == 0) { + open MAP, '<', $ARGV[1] or die "Cannot open $ARGV[1]: $!\n"; + binmode MAP; + { + $/ = undef; + my $map = ; + #baseaddr could also be extracted from PE header + $map =~ /__image_base__ = (0x[0-9a-f]+)/; + my $baseaddr = hex($1); + die "base address of PE file not found" unless $baseaddr; + my @descriptors = $map =~ + / (0x[0-9a-f]+) (?:_DELAY_IMPORT_DESCRIPTOR_|_NULL_DELAY_IMPORT_DESCRIPTOR)/g; + die "no delay loaded libraries found in $ARGV[1]" unless(@descriptors); + #TODO make sure all descriptors are 0x20 apart incase linker's design + #changes in future + $va = hex($descriptors[0])-$baseaddr; + $size = @descriptors*0x20 + } + printf("adding IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT VA=0x%x Size=0x%x", $va, $size); + seek EXE, $offset+0xE0, 0; + print EXE pack "LL", $va, $size; +} else { + die "Found existing delayed import header entry, not continuing"; +} +close EXE; +__END__ + +Visual C crashes reading a dll without the NULL thunk. Theoretically Visual C +should be using +OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT].Size +to iterate +over the array of ImgDelayDescr structs but instead it uses "null termination". + +Other 3rd party PE tools like PE Explorer and Ida throw errors trying to read +the x86 machine code as RVAs/pointers after these ImgDelayDescr structs, so null +termination instead of .Size member seems to the universal implementation for +parsing delay import descriptors. The ImgDelayDescr structs are allocated in +.text section because of dlltool's/ld's implementation (or lack thereof) of +delay loading which explains why there is machine code after the structs. + +C:\perl\src\win32> dumpbin /imports ..\lib\auto\win32\Win32.dll +Microsoft (R) COFF/PE Dumper Version 7.10.6030 +Copyright (C) Microsoft Corporation. All rights reserved. + + +Dump of file ..\lib\auto\win32\Win32.dll + +File Type: DLL + + Section contains the following delay load imports: + + version.dll + 00000001 Characteristics + 70A49008 Address of HMODULE + 70A50514 Import Address Table + 70A502C8 Import Name Table + 00000000 Bound Import Name Table + 00000000 Unload Import Name Table + 0 time date stamp + + 70A47586 0 GetFileVersionInfoA + 70A47576 1 GetFileVersionInfoSizeA + 70A47566 A VerQueryValueA + + ole32.dll + 00000001 Characteristics + 70A4900C Address of HMODULE + 70A50500 Import Address Table + 70A502B4 Import Name Table + 00000000 Bound Import Name Table + 00000000 Unload Import Name Table + 0 time date stamp + + 70A475CA F CoCreateGuid + 70A475BA 6A CoTaskMemFree + 70A475AA 13E StringFromCLSID + + (null) + A11CEC83 Characteristics + D0C804C7 Address of HMODULE + FA14A4A0 Import Address Table + 16FA82444 Import Name Table + 115A6E815 Bound Import Name Table + 101349070 Unload Import Name Table + 90669066 time date stamp + + +DUMPBIN : fatal error LNK1000: Internal error during DumpDelayLoadImports + + Version 7.10.6030 + + ExceptionCode = C0000005 + ExceptionFlags = 00000000 + ExceptionAddress = 0043B42A (00400000) "C:\Program Files\Microsoft Vis +ual Studio .NET 2003\Vc7\bin\link.exe" + NumberParameters = 00000002 + ExceptionInformation[ 0] = 00000000 + ExceptionInformation[ 1] = 7EFDE044 + +CONTEXT: + Eax = FF03E044 Esp = 0012E6E0 + Ebx = FF03E044 Ebp = 0012E7BC + Ecx = 0000DC00 Esi = 011A5AC8 + Edx = 7FFA0000 Edi = 00000000 + Eip = 0043B42A EFlags = 00010286 + SegCs = 0000001B SegDs = 00000023 + SegSs = 00000023 SegEs = 00000023 + SegFs = 0000003B SegGs = 00000000 + Dr0 = 0012E6E0 Dr3 = FF03E044 + Dr1 = 0012E7BC Dr6 = 0000DC00 + Dr2 = 00000000 Dr7 = 00000000 + +C:\perl\src\win32> -- 2.5.0.windows.1 ```
p5pRT commented 6 years ago

From @bulk88

On Wed\, 30 May 2018 23​:20​:22 -0700\, bulk88 wrote​:

Version bumped patch\, but this patch is a working POC. My main goal is getting winsock in libperl delay loaded with GCC.

manifest revised.

-- bulk88 ~ bulk88 at hotmail.com

p5pRT commented 6 years ago

From @bulk88

0001-cpan-Win32-add-delay-loading-for-GCC-and-VC.patch ```diff From 74f179d5ab2ec4446bd53af72b470dbdb51034fb Mon Sep 17 00:00:00 2001 From: Daniel Dragan Date: Tue, 5 Jun 2018 22:37:24 -0400 Subject: [PATCH] cpan/Win32 add delay loading for GCC and VC Most common use of Win32:: is cwd() and short vs long or rel vs abs path functions. Followed by GetLastError. version.dll is very rarely used. ole32.dll loads RPC service registration and COM and registry stuff into the process. To speed up all perl modules EUMM building and later testing. Put these 2 DLLs to be delay loaded. Visual C is easy to delay load with. With GCC it is complicated, with little to no known public use of the feature, and the feature is crudely hacked into LD without LD being aware of what delay loaded DLLs are. Technically, the linker and OS PE loader never need to be aware what delay loaded DLLs are. Its just a pointer table initially all pointing at one var arg, CPU context saving (think setjmp) function written in asm, that loads the DLL, and writes the func ptr into the array, then longjmp()s into the newly fetched function in another DLL. Subsequent calls goto the real function and not the vararg loader one. There is one serious bug with GCC delay loading, described at https://sourceware.org/bugzilla/show_bug.cgi?id=14339 . I have worked around that. And a spec violation that the delay loading structs arent mentioned in the PE header. So static PE analysis tools dont show the delayed imports for GCC, unlike for VC binaries where delayed imports are listed by public PE tools. Delayed imports not being mentioned in the PE header doesn't affect the delayed feature. But it prevents "binding" the delayed imports to OS DLLs (perl doesn't currently do this, maybe one day it will if I publish the code) and it prevents debugging tools from seeing them. To fix this second spec violation, compute the RVA of the delayed import struct array and write it into the PE header after the linker puts the DLL on disk. Use a GCC map file to find the abs addr of the struct inside the DLL. Also add a null termination array slice "nulldlydescr.c" to stop debugging tools from crashing or throwing errors, since VC always generates the null filled struct. I named the symbol _NULL_DELAY_IMPORT_DESCRIPTOR after how VC internally names it, but VC uses a different pattern of symbol names than dlltool does. dlltool is "_DELAY_IMPORT_DESCRIPTOR_ole32_dlya" VC is "__DELAY_IMPORT_DESCRIPTOR_ole32"/"__NULL_DELAY_IMPORT_DESCRIPTOR" --- MANIFEST | 3 + cpan/Win32/.gitignore | 4 ++ cpan/Win32/Makefile.PL | 109 ++++++++++++++++++++++++++++++++ cpan/Win32/Win32.pm | 2 +- cpan/Win32/Win32.xs | 34 ++++++++++ cpan/Win32/mkdlylib.PL | 49 +++++++++++++++ cpan/Win32/nulldlydescr.c | 13 ++++ cpan/Win32/setdlyhdr.PL | 157 ++++++++++++++++++++++++++++++++++++++++++++++ 8 files changed, 370 insertions(+), 1 deletion(-) create mode 100644 cpan/Win32/.gitignore create mode 100644 cpan/Win32/mkdlylib.PL create mode 100644 cpan/Win32/nulldlydescr.c create mode 100644 cpan/Win32/setdlyhdr.PL diff --git a/MANIFEST b/MANIFEST index 2005f54..1ceec7e 100644 --- a/MANIFEST +++ b/MANIFEST @@ -2881,6 +2881,9 @@ cpan/version/t/11_taint.t Tests for version objects cpan/version/t/coretests.pm Tests for version objects cpan/Win32/longpath.inc Win32 extension long path support cpan/Win32/Makefile.PL Win32 extension makefile writer +cpan/Win32/mkdlylib.PL Win32 extension gen delayed lib for GCC +cpan/Win32/nulldlydescr.c empty delay lib descriptor for GCC +cpan/Win32/setdlyhdr.PL Win32 fixup DLL header cpan/Win32/t/CodePage.t See if Win32 extension works cpan/Win32/t/CreateFile.t See if Win32 extension works cpan/Win32/t/ExpandEnvironmentStrings.t See if Win32 extension works diff --git a/cpan/Win32/.gitignore b/cpan/Win32/.gitignore new file mode 100644 index 0000000..c16cbf7 --- /dev/null +++ b/cpan/Win32/.gitignore @@ -0,0 +1,4 @@ +*.tdef +*.dlya +*.s +*.map diff --git a/cpan/Win32/Makefile.PL b/cpan/Win32/Makefile.PL index 0f16594..2ab68ee 100644 --- a/cpan/Win32/Makefile.PL +++ b/cpan/Win32/Makefile.PL @@ -2,17 +2,126 @@ use 5.006; use strict; use warnings; use ExtUtils::MakeMaker; +use Config; unless ($^O eq "MSWin32" || $^O eq "cygwin") { die "OS unsupported\n"; } +#use delay loading for version.dll and ole32.dll +#if true, checks will be performed if delay loading can be done +#if checks fail, delay loading will not be done +my $CanMSVCDelayLoad = 1; +my $CanGCCDelayLoad = 1; + +#set this macro here, additional stuff might be appended +my $OTHERLDFLAGS = ''; + +CheckMSVCDelayLoad(); +CheckGCCDelayLoad(); my %param = ( NAME => 'Win32', VERSION_FROM => 'Win32.pm', INSTALLDIRS => ($] >= 5.008004 && $] < 5.012 ? 'perl' : 'site'), + PL_FILES => {}, + DEFINE => ($CanGCCDelayLoad ? '-DMY_GCC_DELAY_LOAD' : ''), + clean => {FILES => '*.tdef *.dlya *.s *.map'}, + dynamic_lib => { + OTHERLDFLAGS => + $OTHERLDFLAGS . ($CanMSVCDelayLoad ? + ' -DELAYLOAD:version.dll -DELAYLOAD:ole32.dll delayimp.lib ' + : $CanGCCDelayLoad ? ' -Wl,-Map=$(BASEEXT).map ' + : '') + } ); $param{NO_META} = 1 if eval "$ExtUtils::MakeMaker::VERSION" >= 6.10_03; $param{LIBS} = ['-L/lib/w32api -lole32 -lversion'] if $^O eq "cygwin"; WriteMakefile(%param); + +sub CheckMSVCDelayLoad { + my $LocalCanMSVCDelayLoad = 0; + if($CanMSVCDelayLoad && $Config{ld} eq 'link') { + my $linkoutput = `link /?`; + if($linkoutput =~ /delayload/i) { + $LocalCanMSVCDelayLoad = 1; + } + } + $CanMSVCDelayLoad = $LocalCanMSVCDelayLoad; + print "CanMSVCDelayLoad= $CanMSVCDelayLoad\n"; +} + +sub CheckGCCDelayLoad{ + my $LocalCanGCCDelayLoad = 0; #cygwin not tested so dly ld disabled + if($CanGCCDelayLoad && $Config{cc} eq 'gcc' && $^O ne "cygwin") { + my $dlltoolhelp = `dlltool -h`; + my $nm = `nm -V`; + if(index($dlltoolhelp , 'output-delaylib') != -1 + && index($nm, 'GNU General Public License') != -1) { + my $libpaths; + my $cmd = "$Config{cc} -print-search-dirs"; + foreach(`$cmd`) { + if($_ =~ /^libraries: =(.+)$/) { + $libpaths = $1; + last; + } + } + die "command \"$cmd\" failed to return a library search path" + unless $libpaths; + my @libpaths = split(';', $libpaths); +# add compiler default lib paths (not from %Config or env) to EU::LL's psuedo +# lib search dirs since most GCC Win32 builds have CCHOME AKA $Config{libpth} +# set wrong + my @mingwexarr = ExtUtils::Liblist->ext( + join(' ', map(qq["-L$_"], @libpaths)).' -lmingwex :nodefault', + 1, 1 + ); + if($mingwexarr[0] =~ /mingwex/i){ + my $runstr = 'nm -g "'.$mingwexarr[0].'"'; + my $mingwexdump = `$runstr`; + if(index($mingwexdump,'__delayLoadHelper2') != -1){ + $LocalCanGCCDelayLoad = 1; + } + } + } + } + $CanGCCDelayLoad = $LocalCanGCCDelayLoad; + print "CanGCCDelayLoad= $CanGCCDelayLoad\n"; +} + +package MY; + +sub postamble { + return +#note %.dlya is dmake only syntax +($CanGCCDelayLoad ? ' +$(INST_DYNAMIC) : version.dlya ole32.dlya nulldlydescr.o + +%.dlya : mkdlylib.PL + $(PERLRUN) mkdlylib.PL '.$main::Config{cc}.' $* + +':''); + +} + +sub const_loadlibs { + my ($self) = @_; + if($CanGCCDelayLoad) { + #ordering of nulldlydescr.o is very important, must be after all .dlya files + $self->{LDLOADLIBS} .= ' version.dlya ole32.dlya nulldlydescr.o'; + $self->{LDLOADLIBS} =~ s/-lversion//; + $self->{LDLOADLIBS} =~ s/-lole32//; + } + return $self->SUPER::const_loadlibs; +} + +sub dynamic_lib { + my $self = shift; #note OTHERLDFLAGS is in @_ + my $block = $self->SUPER::dynamic_lib(@_); + if($CanGCCDelayLoad) { + substr($block, -1, 1, ' + $(PERLRUN) setdlyhdr.PL $@ $(BASEEXT).map +'); + } + return $block; +} diff --git a/cpan/Win32/Win32.pm b/cpan/Win32/Win32.pm index 7b9ab45..a9126f8 100644 --- a/cpan/Win32/Win32.pm +++ b/cpan/Win32/Win32.pm @@ -8,7 +8,7 @@ package Win32; require DynaLoader; @ISA = qw|Exporter DynaLoader|; - $VERSION = '0.52'; + $VERSION = '0.52_01'; $XS_VERSION = $VERSION; $VERSION = eval $VERSION; diff --git a/cpan/Win32/Win32.xs b/cpan/Win32/Win32.xs index de3764e..69abdcf 100644 --- a/cpan/Win32/Win32.xs +++ b/cpan/Win32/Win32.xs @@ -12,6 +12,40 @@ # define countof(array) (sizeof (array) / sizeof (*(array))) #endif +#ifdef MY_GCC_DELAY_LOAD +# include +/*attribute dllimport causes access vio to integer 6 crash, the _imp_ is corrupt +in the delay load libs dlltool generated, the mingw headers use attribute +dllimport, get rid of attribute dllimport and the bug goes away + +see https://sourceware.org/bugzilla/show_bug.cgi?id=14339 +*/ +# define GCC_VERSION (__GNUC__ * 10000 \ + + __GNUC_MINOR__ * 100 \ + + __GNUC_PATCHLEVEL__) +/*pop push added in gcc 4.6*/ +# if GCC_VERSION >= 40600 +# pragma GCC diagnostic push +# endif +# pragma GCC diagnostic ignored "-Wattributes" +extern HRESULT WINAPI CoCreateGuid(GUID * pguid); +extern HRESULT WINAPI StringFromCLSID(const IID * const rclsid, LPOLESTR * lplpsz); +extern void WINAPI CoTaskMemFree(LPVOID pv); +# if GCC_VERSION >= 40600 +# pragma GCC diagnostic pop +# else +# pragma GCC diagnostic warning "-Wattributes" +# endif +/* test offsets in setdlyhdr.pl */ +# ifdef WIN64 +/* prob wrong */ +STATIC_ASSERT_DECL(STRUCT_OFFSET(IMAGE_NT_HEADERS64,OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT])==0xE0); +# else +STATIC_ASSERT_DECL(STRUCT_OFFSET(IMAGE_NT_HEADERS32,OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT])==0xE0); +# endif +STATIC_ASSERT_DECL(sizeof(ImgDelayDescr) == 0x20); +#endif + #define SE_SHUTDOWN_NAMEA "SeShutdownPrivilege" #ifndef WC_NO_BEST_FIT_CHARS diff --git a/cpan/Win32/mkdlylib.PL b/cpan/Win32/mkdlylib.PL new file mode 100644 index 0000000..5953325 --- /dev/null +++ b/cpan/Win32/mkdlylib.PL @@ -0,0 +1,49 @@ +#!/usr/bin/perl -w +use strict; +require ExtUtils::Liblist; +#file extension meanings, selected for easy makefile cleaning +#tdef = temp def file +#dlya = delay a file + +my $cc = $ARGV[0]; +my $basename = $ARGV[1]; +die "usage: perl mkdlylib.PL cc libbasename" unless $cc && $basename; +my $dotaname = 'lib'.$basename.'.a'; + +my $libpaths; +my $cmd = "$cc -print-search-dirs"; +foreach(`$cmd`) { + if($_ =~ /^libraries: =(.+)$/) { + $libpaths = $1; + last; + } +} +die "command \"$cmd\" failed to return a library search path" unless $libpaths; +my @libpaths = split(';', $libpaths); +my @arr = ExtUtils::Liblist->ext( +# add compiler default lib paths (not from %Config or env) to EU::LL's psuedo +# lib search dirs since most GCC Win32 builds have CCHOME AKA $Config{libpth} +# set wrong + join(' ', map(qq["-L$_"], @libpaths)).' -l'.$basename.' :nodefault' + , 0, 1 +); +my $libpath; +#remove the search loop, just 1 elem now +foreach(@{$arr[4]}) { + if(index($_, $dotaname) != -1){ + $libpath = $_; + last; + } +} +die("ExtUtils::Liblist can't find $dotaname") if ! defined($libpath); +my $nmout =`nm -g $libpath`; +my @exports = $nmout =~ m/^[[:xdigit:]]+ T _*(.+)$/mg; +die "nm failed, output of nm was:\n\n".$nmout if !@exports; +my $defstr = 'LIBRARY '.$basename.'.dll +EXPORTS +'.join("\n", @exports)."\n"; +open(FILE, '>', $basename.'.tdef') or die $!; +print(FILE $defstr) or die $!; +close(FILE) or die $!; +my $retcode = system('dlltool -y '.$basename.'.dlya --input-def '.$basename.'.tdef'); +die "dlltool failed with exit code ".($retcode >> 8) if $retcode != 0; diff --git a/cpan/Win32/nulldlydescr.c b/cpan/Win32/nulldlydescr.c new file mode 100644 index 0000000..6264e82 --- /dev/null +++ b/cpan/Win32/nulldlydescr.c @@ -0,0 +1,13 @@ +/* Dont bother with headers, just make a zero filled ImgDelayDescr struct. + * This file is only used with GCC delay loading, to correct the array of + * ImgDelayDescr struct to meet Visual C linker/PE debugging tool standards + */ +char _NULL_DELAY_IMPORT_DESCRIPTOR [0x20] + __attribute__ ((section (".text$2"))) +/* GCC by default will create the object file section with 32 byte alignment + probably because this object is 32 bytes long but all ImgDelayDescr structs + created by dlltool have 4 byte align, and we dont want the linker to put + padding (aka uninit data) between ImgDelayDescr structs +*/ + __attribute__ ((aligned (4))) + = {}; diff --git a/cpan/Win32/setdlyhdr.PL b/cpan/Win32/setdlyhdr.PL new file mode 100644 index 0000000..5d0dca8 --- /dev/null +++ b/cpan/Win32/setdlyhdr.PL @@ -0,0 +1,157 @@ +#!perl -w +use strict; +use Data::Dumper; +#derived from Perl core win32/bin/exetype.pl + +# All the IMAGE_* structures are defined in the WINNT.H file +# of the Microsoft Platform SDK. + +unless (0 < @ARGV && @ARGV < 3) { + print "Usage: $0 dllexefile mapfile\n"; + exit 1; +} + +my ($record,$magic,$signature,$offset,$va, $size, $win64); +open EXE, '+<', $ARGV[0] or die "Cannot open $ARGV[0]: $!\n"; +binmode EXE; + +# read IMAGE_DOS_HEADER structure +read EXE, $record, 64; +($magic,$offset) = unpack "Sx58L", $record; + +die "$ARGV[0] is not an MSDOS executable file.\n" + unless $magic == 0x5a4d; # "MZ" + +# read signature, IMAGE_FILE_HEADER and first WORD of IMAGE_OPTIONAL_HEADER +seek EXE, $offset, 0; +read EXE, $record, 4+20+2; +($signature,$size,$magic) = unpack "Lx16Sx2S", $record; + +die "PE header not found" unless $signature == 0x4550; # "PE\0\0" + +if($size == 224 && $magic == 0x10b) { # IMAGE_NT_OPTIONAL_HDR32_MAGIC + $win64 = 0; +} elsif ($size == 240 && $magic == 0x20b) { # IMAGE_NT_OPTIONAL_HDR64_MAGIC + $win64 = 1; +} else { + die "Optional header is neither in NT32 nor in NT64 format"; +} + +# Offset 0xE0 in the IMAGE_OPTIONAL_HEADER(32|64) is +# OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT] +seek EXE, $offset+0xE0, 0; +read EXE, $record, 8; +($va,$size) = unpack "LL", $record; +if ($va == 0 && $size == 0) { + open MAP, '<', $ARGV[1] or die "Cannot open $ARGV[1]: $!\n"; + binmode MAP; + { + $/ = undef; + my $map = ; + #baseaddr could also be extracted from PE header + $map =~ /__image_base__ = (0x[0-9a-f]+)/; + my $baseaddr = hex($1); + die "base address of PE file not found" unless $baseaddr; + my @descriptors = $map =~ + / (0x[0-9a-f]+) (?:_DELAY_IMPORT_DESCRIPTOR_|_NULL_DELAY_IMPORT_DESCRIPTOR)/g; + die "no delay loaded libraries found in $ARGV[1]" unless(@descriptors); + #TODO make sure all descriptors are 0x20 apart incase linker's design + #changes in future + $va = hex($descriptors[0])-$baseaddr; + $size = @descriptors*0x20 + } + printf("adding IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT VA=0x%x Size=0x%x", $va, $size); + seek EXE, $offset+0xE0, 0; + print EXE pack "LL", $va, $size; +} else { + die "Found existing delayed import header entry, not continuing"; +} +close EXE; +__END__ + +Visual C crashes reading a dll without the NULL thunk. Theoretically Visual C +should be using +OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_DELAY_IMPORT].Size +to iterate +over the array of ImgDelayDescr structs but instead it uses "null termination". + +Other 3rd party PE tools like PE Explorer and Ida throw errors trying to read +the x86 machine code as RVAs/pointers after these ImgDelayDescr structs, so null +termination instead of .Size member seems to the universal implementation for +parsing delay import descriptors. The ImgDelayDescr structs are allocated in +.text section because of dlltool's/ld's implementation (or lack thereof) of +delay loading which explains why there is machine code after the structs. + +C:\perl\src\win32> dumpbin /imports ..\lib\auto\win32\Win32.dll +Microsoft (R) COFF/PE Dumper Version 7.10.6030 +Copyright (C) Microsoft Corporation. All rights reserved. + + +Dump of file ..\lib\auto\win32\Win32.dll + +File Type: DLL + + Section contains the following delay load imports: + + version.dll + 00000001 Characteristics + 70A49008 Address of HMODULE + 70A50514 Import Address Table + 70A502C8 Import Name Table + 00000000 Bound Import Name Table + 00000000 Unload Import Name Table + 0 time date stamp + + 70A47586 0 GetFileVersionInfoA + 70A47576 1 GetFileVersionInfoSizeA + 70A47566 A VerQueryValueA + + ole32.dll + 00000001 Characteristics + 70A4900C Address of HMODULE + 70A50500 Import Address Table + 70A502B4 Import Name Table + 00000000 Bound Import Name Table + 00000000 Unload Import Name Table + 0 time date stamp + + 70A475CA F CoCreateGuid + 70A475BA 6A CoTaskMemFree + 70A475AA 13E StringFromCLSID + + (null) + A11CEC83 Characteristics + D0C804C7 Address of HMODULE + FA14A4A0 Import Address Table + 16FA82444 Import Name Table + 115A6E815 Bound Import Name Table + 101349070 Unload Import Name Table + 90669066 time date stamp + + +DUMPBIN : fatal error LNK1000: Internal error during DumpDelayLoadImports + + Version 7.10.6030 + + ExceptionCode = C0000005 + ExceptionFlags = 00000000 + ExceptionAddress = 0043B42A (00400000) "C:\Program Files\Microsoft Vis +ual Studio .NET 2003\Vc7\bin\link.exe" + NumberParameters = 00000002 + ExceptionInformation[ 0] = 00000000 + ExceptionInformation[ 1] = 7EFDE044 + +CONTEXT: + Eax = FF03E044 Esp = 0012E6E0 + Ebx = FF03E044 Ebp = 0012E7BC + Ecx = 0000DC00 Esi = 011A5AC8 + Edx = 7FFA0000 Edi = 00000000 + Eip = 0043B42A EFlags = 00010286 + SegCs = 0000001B SegDs = 00000023 + SegSs = 00000023 SegEs = 00000023 + SegFs = 0000003B SegGs = 00000000 + Dr0 = 0012E6E0 Dr3 = FF03E044 + Dr1 = 0012E7BC Dr6 = 0000DC00 + Dr2 = 00000000 Dr7 = 00000000 + +C:\perl\src\win32> -- 2.5.0.windows.1 ```