Open p5pRT opened 7 years ago
p5pRT
commented
7 years ago I have a number of modules which I've installed via 'cpanm'. Sometimes when I upgrade Perl I get an error message like this:
$ some-script-of-mine XS.c: loadable library and perl binaries are mismatched (got handshake key 0x7ec0080\, needed 0x7f00080)
This message would be more helpful to me if it specified the module which is being loaded when the message is generated.
p5pRT
commented
7 years ago On Sat\, 04 Feb 2017 23:08:09 GMT\, frederik@ofb.net wrote:
This is a bug report for perl from frederik@ofb.net\, generated with the help of perlbug 1.40 running under perl 5.24.0.
----------------------------------------------------------------- [Please describe your issue here]
I have a number of modules which I've installed via 'cpanm'. Sometimes when I upgrade Perl I get an error message like this:
$ some-script-of-mine XS.c: loadable library and perl binaries are mismatched (got handshake key 0x7ec0080\, needed 0x7f00080)
This message would be more helpful to me if it specified the module which is being loaded when the message is generated.
Does the discussion in https://rt-archive.perl.org/perl5/Ticket/Display.html?id=124201 help you understand the problem better?
Thank you very much.
-- James E Keenan (jkeenan@cpan.org)
p5pRT
commented
7 years ago The RT System itself - Status changed from 'new' to 'open'
p5pRT
commented
7 years ago On Sat\, 04 Feb 2017 23:08:09 GMT\, frederik@ofb.net wrote:
This is a bug report for perl from frederik@ofb.net\, generated with the help of perlbug 1.40 running under perl 5.24.0.
----------------------------------------------------------------- [Please describe your issue here]
I have a number of modules which I've installed via 'cpanm'. Sometimes when I upgrade Perl I get an error message like this:
$ some-script-of-mine XS.c: loadable library and perl binaries are mismatched (got handshake key 0x7ec0080\, needed 0x7f00080)
This message would be more helpful to me if it specified the module which is being loaded when the message is generated.
Also\, can you supply the output of 'perl -V' for both the version of perl against which your module was originally compiled\, as well as that for the perl to which you upgraded (assuming that the 'perl -V' attached to this ticket is neither)?
Thank you very much. Jim Keenan
[Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=low --- Site configuration information for perl 5.24.0:
Configured by builduser at Thu Sep 8 16:15:56 CEST 2016.
Summary of my perl5 (revision 5 version 24 subversion 0) configuration:
Platform: osname=linux\, osvers=4.7.3-1-arch\, archname=i686-linux-thread-multi uname='linux flo-32 4.7.3-1-arch #1 smp preempt wed sep 7 17:57:38 cest 2016 i686 gnulinux ' config_args='-des -Dusethreads -Duseshrplib -Doptimize=-march=i686 -mtune=generic -O2 -pipe -fstack-protector-strong -Dprefix=/usr -Dvendorprefix=/usr -Dprivlib=/usr/share/perl5/core_perl -Darchlib=/usr/lib/perl5/core_perl -Dsitelib=/usr/share/perl5/site_perl -Dsitearch=/usr/lib/perl5/site_perl -Dvendorlib=/usr/share/perl5/vendor_perl -Dvendorarch=/usr/lib/perl5/vendor_perl -Dscriptdir=/usr/bin/core_perl -Dsitescript=/usr/bin/site_perl -Dvendorscript=/usr/bin/vendor_perl -Dinc_version_list=none -Dman1ext=1perl -Dman3ext=3perl -Dlddlflags=- shared -Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro -Dldflags=-Wl\,-O1\,-- sort-common\,--as-needed\,-z\,relro' hint=recommended\, useposix=true\, d_sigaction=define useithreads=define\, usemultiplicity=define use64bitint=undef\, use64bitall=undef\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='cc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict- aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-march=i686 -mtune=generic -O2 -pipe -fstack-protector- strong'\, cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion=''\, gccversion='6.1.1 20160802'\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234\, doublekind=3 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12\, longdblkind=3 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=4\, prototype=define Linker and Libraries: ld='cc'\, ldflags ='-Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/i686-pc-linux-gnu/6.1.1/include- fixed /usr/lib /lib libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.24.so\, so=so\, useshrplib=true\, libperl=libperl.so gnulibc_version='2.24' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-rpath\,/usr/lib/perl5/core_perl/CORE' cccdlflags='-fPIC'\, lddlflags='-shared -Wl\,-O1\,--sort-common\,--as- needed\,-z\,relro -L/usr/local/lib -fstack-protector-strong'
--- @INC for perl 5.24.0: /home/frederik/scripts-misc/perl /home/frederik/.local/lib/perl5/i686-linux-thread-multi /home/frederik/.local/lib/perl5 /usr/lib/perl5/site_perl /usr/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl .
--- Environment for perl 5.24.0: HOME=/home/frederik LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH=/home/frederik/.local/arch/i386/lib:/home/frederik/.local/lib:/usr/local/lib LOGDIR (unset) PATH=/home/frederik/.local/bin:/home/frederik/scripts- misc:/home/frederik/.local/arch/i386/bin:/usr/bin/core_perl:/usr/bin/vendor_perl:/usr/bin/site_perl:/usr/local/bin:/usr/local/sbin:/usr/bin PERL5LIB=/home/frederik/scripts- misc/perl:/home/frederik/.local/lib/perl5: PERL_BADLANG (unset) PERL_LOCAL_LIB_ROOT=/home/frederik/.local/:/home/frederik/.local/ PERL_MB_OPT=--install_base "/home/frederik/.local/" PERL_MM_OPT=INSTALL_BASE=/home/frederik/.local/ SHELL=/bin/zsh
-- James E Keenan (jkeenan@cpan.org)
p5pRT
commented
7 years ago Thank you James. Your link describes the problem but I think I understood it already?
I'm just making a feature request for a better error message. TBH\, I could catch the module name with 'strace' or looking in the script's 'use' statements\, but these things stretch my tired brain a bit\, and I imagine lazier users or harder situations exist.
I run Arch and I'm attaching 'perl -V' on the old system (same as generated the bug report) and on an upgraded system. But the upgraded system is 64 bit and the non-upgraded system is 32 bit\, I hope that's not a problem for you. Thanks.
On Sat\, Feb 04\, 2017 at 04:27:55PM -0800\, James E Keenan via RT wrote:
On Sat\, 04 Feb 2017 23:08:09 GMT\, frederik@ofb.net wrote:
This is a bug report for perl from frederik@ofb.net\, generated with the help of perlbug 1.40 running under perl 5.24.0.
----------------------------------------------------------------- [Please describe your issue here]
I have a number of modules which I've installed via 'cpanm'. Sometimes when I upgrade Perl I get an error message like this:
$ some-script-of-mine XS.c: loadable library and perl binaries are mismatched (got handshake key 0x7ec0080\, needed 0x7f00080)
This message would be more helpful to me if it specified the module which is being loaded when the message is generated.
Also\, can you supply the output of 'perl -V' for both the version of perl against which your module was originally compiled\, as well as that for the perl to which you upgraded (assuming that the 'perl -V' attached to this ticket is neither)?
Thank you very much. Jim Keenan
[Please do not change anything below this line] ----------------------------------------------------------------- --- Flags: category=core severity=low --- Site configuration information for perl 5.24.0:
Configured by builduser at Thu Sep 8 16:15:56 CEST 2016.
Summary of my perl5 (revision 5 version 24 subversion 0) configuration:
Platform: osname=linux\, osvers=4.7.3-1-arch\, archname=i686-linux-thread-multi uname='linux flo-32 4.7.3-1-arch #1 smp preempt wed sep 7 17:57:38 cest 2016 i686 gnulinux ' config_args='-des -Dusethreads -Duseshrplib -Doptimize=-march=i686 -mtune=generic -O2 -pipe -fstack-protector-strong -Dprefix=/usr -Dvendorprefix=/usr -Dprivlib=/usr/share/perl5/core_perl -Darchlib=/usr/lib/perl5/core_perl -Dsitelib=/usr/share/perl5/site_perl -Dsitearch=/usr/lib/perl5/site_perl -Dvendorlib=/usr/share/perl5/vendor_perl -Dvendorarch=/usr/lib/perl5/vendor_perl -Dscriptdir=/usr/bin/core_perl -Dsitescript=/usr/bin/site_perl -Dvendorscript=/usr/bin/vendor_perl -Dinc_version_list=none -Dman1ext=1perl -Dman3ext=3perl -Dlddlflags=- shared -Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro -Dldflags=-Wl\,-O1\,-- sort-common\,--as-needed\,-z\,relro' hint=recommended\, useposix=true\, d_sigaction=define useithreads=define\, usemultiplicity=define use64bitint=undef\, use64bitall=undef\, uselongdouble=undef usemymalloc=n\, bincompat5005=undef Compiler: cc='cc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict- aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\, optimize='-march=i686 -mtune=generic -O2 -pipe -fstack-protector- strong'\, cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include' ccversion=''\, gccversion='6.1.1 20160802'\, gccosandvers='' intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234\, doublekind=3 d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12\, longdblkind=3 ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8 alignbytes=4\, prototype=define Linker and Libraries: ld='cc'\, ldflags ='-Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro -fstack-protector-strong -L/usr/local/lib' libpth=/usr/local/lib /usr/lib/gcc/i686-pc-linux-gnu/6.1.1/include- fixed /usr/lib /lib libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc libc=libc-2.24.so\, so=so\, useshrplib=true\, libperl=libperl.so gnulibc_version='2.24' Dynamic Linking: dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-rpath\,/usr/lib/perl5/core_perl/CORE' cccdlflags='-fPIC'\, lddlflags='-shared -Wl\,-O1\,--sort-common\,--as- needed\,-z\,relro -L/usr/local/lib -fstack-protector-strong'
--- @INC for perl 5.24.0: /home/frederik/scripts-misc/perl /home/frederik/.local/lib/perl5/i686-linux-thread-multi /home/frederik/.local/lib/perl5 /usr/lib/perl5/site_perl /usr/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl .
--- Environment for perl 5.24.0: HOME=/home/frederik LANG=en_US.UTF-8 LANGUAGE (unset) LD_LIBRARY_PATH=/home/frederik/.local/arch/i386/lib:/home/frederik/.local/lib:/usr/local/lib LOGDIR (unset) PATH=/home/frederik/.local/bin:/home/frederik/scripts- misc:/home/frederik/.local/arch/i386/bin:/usr/bin/core_perl:/usr/bin/vendor_perl:/usr/bin/site_perl:/usr/local/bin:/usr/local/sbin:/usr/bin PERL5LIB=/home/frederik/scripts- misc/perl:/home/frederik/.local/lib/perl5: PERL_BADLANG (unset) PERL_LOCAL_LIB_ROOT=/home/frederik/.local/:/home/frederik/.local/ PERL_MB_OPT=--install_base "/home/frederik/.local/" PERL_MM_OPT=INSTALL_BASE=/home/frederik/.local/ SHELL=/bin/zsh
-- James E Keenan (jkeenan@cpan.org)
p5pRT
commented
7 years ago Summary of my perl5 (revision 5 version 24 subversion 0) configuration:
Platform:
osname=linux\, osvers=4.7.3-1-arch\, archname=i686-linux-thread-multi
uname='linux flo-32 4.7.3-1-arch #1 smp preempt wed sep 7 17:57:38 cest 2016 i686 gnulinux '
config_args='-des -Dusethreads -Duseshrplib -Doptimize=-march=i686 -mtune=generic -O2 -pipe -fstack-protector-strong -Dprefix=/usr -Dvendorprefix=/usr -Dprivlib=/usr/share/perl5/core_perl -Darchlib=/usr/lib/perl5/core_perl -Dsitelib=/usr/share/perl5/site_perl -Dsitearch=/usr/lib/perl5/site_perl -Dvendorlib=/usr/share/perl5/vendor_perl -Dvendorarch=/usr/lib/perl5/vendor_perl -Dscriptdir=/usr/bin/core_perl -Dsitescript=/usr/bin/site_perl -Dvendorscript=/usr/bin/vendor_perl -Dinc_version_list=none -Dman1ext=1perl -Dman3ext=3perl -Dlddlflags=-shared -Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro -Dldflags=-Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro'
hint=recommended\, useposix=true\, d_sigaction=define
useithreads=define\, usemultiplicity=define
use64bitint=undef\, use64bitall=undef\, uselongdouble=undef
usemymalloc=n\, bincompat5005=undef
Compiler:
cc='cc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\,
optimize='-march=i686 -mtune=generic -O2 -pipe -fstack-protector-strong'\,
cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''\, gccversion='6.1.1 20160802'\, gccosandvers=''
intsize=4\, longsize=4\, ptrsize=4\, doublesize=8\, byteorder=1234\, doublekind=3
d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=12\, longdblkind=3
ivtype='long'\, ivsize=4\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8
alignbytes=4\, prototype=define
Linker and Libraries:
ld='cc'\, ldflags ='-Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/i686-pc-linux-gnu/6.1.1/include-fixed /usr/lib /lib
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.24.so\, so=so\, useshrplib=true\, libperl=libperl.so
gnulibc_version='2.24'
Dynamic Linking:
dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-rpath\,/usr/lib/perl5/core_perl/CORE'
cccdlflags='-fPIC'\, lddlflags='-shared -Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro -L/usr/local/lib -fstack-protector-strong'
Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP PERL_PRESERVE_IVUV USE_ITHREADS USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API Built under linux Compiled at Sep 8 2016 16:16:30 %ENV: PERL5LIB="/home/frederik/scripts-misc/perl:/home/frederik/.local/lib/perl5:" PERL_LOCAL_LIB_ROOT="/home/frederik/.local/" PERL_MB_OPT="--install_base "/home/frederik/.local/"" PERL_MM_OPT="INSTALL_BASE=/home/frederik/.local/" @INC: /home/frederik/scripts-misc/perl /home/frederik/.local/lib/perl5/i686-linux-thread-multi /home/frederik/.local/lib/perl5 /usr/lib/perl5/site_perl /usr/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl .
p5pRT
commented
7 years ago Summary of my perl5 (revision 5 version 24 subversion 1) configuration:
Platform:
osname=linux\, osvers=4.9.3-1-arch\, archname=x86_64-linux-thread-multi
uname='linux flo-64 4.9.3-1-arch #1 smp preempt thu jan 12 21:34:12 cet 2017 x86_64 gnulinux '
config_args='-des -Dusethreads -Duseshrplib -Doptimize=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong -Dprefix=/usr -Dvendorprefix=/usr -Dprivlib=/usr/share/perl5/core_perl -Darchlib=/usr/lib/perl5/core_perl -Dsitelib=/usr/share/perl5/site_perl -Dsitearch=/usr/lib/perl5/site_perl -Dvendorlib=/usr/share/perl5/vendor_perl -Dvendorarch=/usr/lib/perl5/vendor_perl -Dscriptdir=/usr/bin/core_perl -Dsitescript=/usr/bin/site_perl -Dvendorscript=/usr/bin/vendor_perl -Dinc_version_list=none -Dman1ext=1perl -Dman3ext=3perl -Dcccdlflags='-fPIC' -Dlddlflags=-shared -Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro -Dldflags=-Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro'
hint=recommended\, useposix=true\, d_sigaction=define
useithreads=define\, usemultiplicity=define
use64bitint=define\, use64bitall=define\, uselongdouble=undef
usemymalloc=n\, bincompat5005=undef
Compiler:
cc='cc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\,
optimize='-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong'\,
cppflags='-D_REENTRANT -D_GNU_SOURCE -fwrapv -fno-strict-aliasing -pipe -fstack-protector-strong -I/usr/local/include'
ccversion=''\, gccversion='6.3.1 20170109'\, gccosandvers=''
intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678\, doublekind=3
d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16\, longdblkind=3
ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8
alignbytes=8\, prototype=define
Linker and Libraries:
ld='cc'\, ldflags ='-Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib/gcc/x86_64-pc-linux-gnu/6.3.1/include-fixed /usr/lib /lib/../lib /usr/lib/../lib /lib /lib64 /usr/lib64
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.24.so\, so=so\, useshrplib=true\, libperl=libperl.so
gnulibc_version='2.24'
Dynamic Linking:
dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E -Wl\,-rpath\,/usr/lib/perl5/core_perl/CORE'
cccdlflags='-fPIC'\, lddlflags='-shared -Wl\,-O1\,--sort-common\,--as-needed\,-z\,relro -L/usr/local/lib -fstack-protector-strong'
Characteristics of this binary (from libperl): Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS PERL_COPY_ON_WRITE PERL_DONT_CREATE_GVSV PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP PERL_PRESERVE_IVUV USE_64_BIT_ALL USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE USE_LOCALE_NUMERIC USE_LOCALE_TIME USE_PERLIO USE_PERL_ATOF USE_REENTRANT_API Built under linux Compiled at Jan 15 2017 14:53:40 %ENV: PERL5LIB="/home/frederik/scripts-misc/perl:/home/frederik/.local/lib/perl5:" PERL_LOCAL_LIB_ROOT="/home/frederik/.local/:/home/frederik/.local/:/home/frederik/.local/" PERL_MB_OPT="--install_base "/home/frederik/.local/"" PERL_MM_OPT="INSTALL_BASE=/home/frederik/.local/" @INC: /home/frederik/scripts-misc/perl /home/frederik/.local/lib/perl5/x86_64-linux-thread-multi /home/frederik/.local/lib/perl5 /usr/lib/perl5/site_perl /usr/share/perl5/site_perl /usr/lib/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib/perl5/core_perl /usr/share/perl5/core_perl .
p5pRT
commented
7 years ago On Sat\, 04 Feb 2017 18:06:42 -0800\, frederik@ofb.net wrote:
Thank you James. Your link describes the problem but I think I understood it already?
I'm just making a feature request for a better error message. TBH\, I could catch the module name with 'strace' or looking in the script's 'use' statements\, but these things stretch my tired brain a bit\, and I imagine lazier users or harder situations exist.
I run Arch and I'm attaching 'perl -V' on the old system (same as generated the bug report) and on an upgraded system. But the upgraded system is 64 bit and the non-upgraded system is 32 bit\, I hope that's not a problem for you. Thanks.
This might be a (conceptual) bug in Perl_xs_handshake().
The XS handshake uses the size of the interpreter structure as part of a "signature" for the perl build\, assuming that a change in size means that the versions of perl are incompatible.
But maintenance releases *can* safely add members to the end of the interpreter structure\, and 2bfbbbaf9ef1783ba914ff9e9270e877fbbb6aba does just that.
The handshake values in frederick's error message:
(got handshake key 0x7ec0080\, needed 0x7f00080)
extract the structure sizes:
got 0x7ec
needed 0x7f0
which differ in sizeof(STRLEN) on the 32-bit system in the original -V output.
So I think Perl_xs_handshake() should only be checking that the size has not shrunk in maintenance releases.
One problem with that is it signifcantly reduces the strength of that check - Perl_xs_handshake() would no longer be detecting the type of problems we want it to detect - differences in sizes of the interpreter structure due to changes to the sizes of it's members (eg use64bitint changes) or due to build options (PERL_HASH_RANDOMIZE_KEYS for example) which may weaken the current interface to near uselessness.
Tony
p5pRT
commented
7 years ago On Sat\, Feb 04\, 2017 at 03:08:09PM -0800\, via RT wrote:
I have a number of modules which I've installed via 'cpanm'. Sometimes when I upgrade Perl I get an error message like this:
$ some\-script\-of\-mine XS\.c​: loadable library and perl binaries are mismatched \(got handshake key 0x7ec0080\, needed 0x7f00080\)This message would be more helpful to me if it specified the module which is being loaded when the message is generated.
Leaving aside the discussions elsewhere in this thread about the handshake key\, as regards displaying the name if the module being loaded\, it already does\, kinda.
XS\.c​: loadable library and perl binaries are mismatched \.\.\.
XS.c is the name of the src file from which the loadable lib was compiled; which isn't same as the name of the perl module\, but may hint at it; for example:
$ ./perl -I../perl-5.24.1t/lib -MPerlIO::scalar -e 1 scalar.c: loadable library and perl binaries are mismatched ... $
The error is given at the point when a dynamic library is loaded; this *may* have been triggered by a use/require\, but not necessary. I suppose we could examine the context stack for an active require\, and if so include that in the error message\, e.g.
while loading MPerlIO::scalar: scalar.c: loadable library and perl binaries are mismatched ...
-- The Enterprise's efficient long-range scanners detect a temporal vortex distortion in good time\, allowing it to be safely avoided via a minor course correction. -- Things That Never Happen in "Star Trek" #21
p5pRT
commented
7 years ago The error is given at the point when a dynamic library is loaded; this *may* have been triggered by a use/require\, but not necessary. I suppose we could examine the context stack for an active require\, and if so include that in the error message\, e.g.
while loading MPerlIO​::scalar​: scalar\.c​: loadable library and perl binaries are mismatched \.\.\.
That's more or less what I had in mind. Up to you if it's too much work / complexity. Would it be easier to just print the full path to the dynamic library\, which would reveal both the module name and where it is installed?
I guess one could argue that I should just have a command to recompile all the locally-installed modules which need recompilation\, whenever I upgrade Perl; since it may not be workable to test each of my scripts individually to see if it produces this message. Module dependencies are handled automatically by my distro package manager\, but how to do this recompilation with e.g. all 'cpanm'-installed modules\, I don't know.
Thanks\,
Frederick
p5pRT
commented
7 years ago On Mon\, Feb 6\, 2017 at 12:53 PM\, \frederik@​ofb\.net wrote:
I guess one could argue that I should just have a command to recompile all the locally-installed modules which need recompilation\, whenever I upgrade Perl; since it may not be workable to test each of my scripts individually to see if it produces this message. Module dependencies are handled automatically by my distro package manager\, but how to do this recompilation with e.g. all 'cpanm'-installed modules\, I don't know.
Thanks\,
Frederick
Generally\, the preferred approach is not installing to a package-managed perl using CPAN.pm or cpanm\, and instead installing to a local::lib which can be blown-away and reinstalled easily upon upgrading perl. (Or by installing your own separate perl for development use\, using perl-build\, perlbrew\, or plenv.) CPAN.pm and cpanm both default to installing to a local::lib if run without write permission to the perl's lib directories\, it's just a matter of setting it up for your user\, usually involving adding a line to .bashrc.
-Dan
p5pRT
commented
7 years ago On Mon\, Feb 6\, 2017 at 12:30 PM\, Dave Mitchell \davem@​iabyn\.com wrote:
XS\.c​: loadable library and perl binaries are mismatched \.\.\.XS.c is the name of the src file from which the loadable lib was compiled; which isn't same as the name of the perl module\, but may hint at it; for example:
$ \./perl \-I\.\./perl\-5\.24\.1t/lib \-MPerlIO​::scalar \-e 1 scalar\.c​: loadable library and perl binaries are mismatched \.\.\. $The error is given at the point when a dynamic library is loaded; this *may* have been triggered by a use/require\, but not necessary. I suppose we could examine the context stack for an active require\, and if so include that in the error message\, e.g.
while loading MPerlIO​::scalar​: scalar\.c​: loadable library and perlbinaries are mismatched ...
The bootstrap function already received the arguments to DynaLoader/XSLoader\, and xs_handshake has access to the stack (even if it's awkward).
Leon
p5pRT
commented
7 years ago On Mon\, 6 Feb 2017 09:53:13 -0800\, frederik@ofb.net said:
I guess one could argue that I should just have a command to recompile all the locally-installed modules which need recompilation\, whenever I upgrade Perl;
CPAN.pm has a command for that. It's called 'recompile'.
-- andreas
p5pRT
commented
7 years ago On Mon\, Feb 06\, 2017 at 07:54:09PM +0100\, Andreas Koenig wrote:
On Mon\, 6 Feb 2017 09:53:13 -0800\, frederik@ofb.net said:
I guess one could argue that I should just have a command to recompile all the locally-installed modules which need recompilation\, whenever I upgrade Perl;
CPAN.pm has a command for that. It's called 'recompile'.
I knew about CPAN::recompile but I had thought I would need a cpanminus-specific solution. Now I find that it seems to work with my setup. I don't think I've configured CPAN other than that I have something in ~/.zprofile like this:
eval $(perl -Mlocal::lib=.local)
I ran the command
perl -MCPAN -e 'recompile()'
and after fetching metadata it seems to be doing the recompilation without any trouble (aside from very occasionally stopping and asking questions).
So another possibility for the error message would be to have it suggest the user recompile all local modules\, and point them to CPAN(3perl)...?
Thanks\,
Frederick
swajime
commented
4 years ago Thank you for this: perl -MCPAN -e 'recompile()'
That was hard to find ...
Upgrading to Ubuntu 20.04 from 18.04 wreaked havoc on my Perl setup.
PrincessRTFM
commented
4 years ago Same problem, but command perl -MCPAN -e 'recompile()' fails with ListUtil.c: loadable library and perl binaries are mismatched (got handshake key 0xde00080, needed 0xcd00080).
swajime
commented
4 years ago Found more info. For an offending package: perldoc -l Moose # offending package is in my .local cpanm --uninstall Moose # removes offending package feom .local perldoc -l Moose # updated package is already in root system per ubuntu upgrade If not, you can install it via apt: aptitude search Moose sudo apt-get install libmoose-perl
I had to do this for a handful of modules, and then I was fine.
If you don't know which module is bad do a find: find / -iname ListUtil.c 2>/dev/null
PrincessRTFM
commented
4 years ago I wound up renaming my ~/perl5 folder, installing App::MigrateModules via cpan, and using perl-migrate-modules --from ~/perl5broken /usr/bin/perl to fix it. Hope that helps someone else!
aditi17142
commented
3 years ago Hi! I updated my OS from Ubuntu 16 to Ubuntu 18 While using the tools, the below error is showing up
Damn.c: loadable library and perl binaries are mismatched (got handshake key 0xdb00080, needed 0xde00080)
Also, I was trying to implement the solution mentioned by @princessRTFM. While doing that accidently deleted the perl folder. Though the executable file "perl" is witnessed in /usr/local. I tried mending it and its not working. Please guide how to go about it.
Thank you.
novacadian
commented
3 years ago Just wanted to mention that this problem had me stumped for some time when working through the process of setting up a Lightning Node. It was finally overcome when using the command "sudo ./autogen.sh" as opposed to "./autogen.sh". Obviously there was some permission issue deep in the Perl libraries.
This solution may be unique to my Ubuntu setup. It's mentioned as a possible last resort solution.
tonycoz
commented
3 years ago I think it could be improved by having EU::PXS creating a macro with the module name (as from the MODULE keyword) and combining that with the FILE to supply to Perl_xs_handshake() instead of just FILE.
It would require untangling the maze of macros in XSUB.h though to define new macros that accept the new string instead of just supplying FILE.
barracuda156
commented
2 years ago I get this error when trying to upgrade help2man:
gettext.c: loadable library and perl binaries are mismatched (got handshake key 0xcf00080, needed 0xce40080)
patch-work
commented
1 month ago > uname -a
OpenBSD server.local 7.6 GENERIC.MP#338 amd64This is a fresh installation:
> doas pkg_add p5-Mail-SpamAssassin
quirks-7.50 signed on 2024-10-15T23:10:48Z
p5-Mail-SpamAssassin-4.0.1:libksba-1.6.7: ok
p5-Mail-SpamAssassin-4.0.1:libassuan-2.5.5: ok
p5-Mail-SpamAssassin-4.0.1:npth-1.6p0: ok
p5-Mail-SpamAssassin-4.0.1:libsecret-0.21.4: ok
p5-Mail-SpamAssassin-4.0.1:pinentry-1.3.0: ok
p5-Mail-SpamAssassin-4.0.1:gnupg-2.4.5: ok
p5-Mail-SpamAssassin-4.0.1:p5-Encode-Detect-1.01p8: ok
p5-Mail-SpamAssassin-4.0.1:p5-IO-String-1.08p3: ok
p5-Mail-SpamAssassin-4.0.1:re2c-3.1: ok
p5-Mail-SpamAssassin-4.0.1:p5-Net-LibIDN2-1.02: ok
p5-Mail-SpamAssassin-4.0.1:p5-Net-CIDR-Lite-0.22: ok
p5-Mail-SpamAssassin-4.0.1:p5-Net-Patricia-1.22p2: ok
p5-Mail-SpamAssassin-4.0.1:p5-BSD-Resource-1.2911p0: ok
p5-Mail-SpamAssassin-4.0.1:p5-Archive-Zip-1.68: ok
useradd: Warning: home directory `/var/db/spamassassin' doesn't exist, and -m was not specified
p5-Mail-SpamAssassin-4.0.1: ok
The following new rcscripts were installed: /etc/rc.d/spamassassin
See rcctl(8) for details.
New and changed readme(s):
/usr/local/share/doc/pkg-readmes/gnupg
/usr/local/share/doc/pkg-readmes/p5-Mail-SpamAssassin
> doas /usr/local/bin/spamd --help | head -4
SpamAssassin Server version 4.0.1
running on Perl 5.38.2
with SSL support (IO::Socket::SSL 2.089)
with zlib support (Compress::Zlib 2.204)OK
This is on a system with previous installations of perl stuff:
> doas /usr/local/bin/spamd --help
Encode.c: loadable library and perl binaries are mismatched (got first handshake key 0xeb80000, needed 0xf280000)So, there is a conflict, but the error message does not help tracking down the root cause of it.
mauke
commented
1 month ago I haven't read the rest of the ticket, but your error message says "Encode.c: loadable library and perl binaries are mismatched", so the Encode module on your system was compiled for a different (incompatible) version of perl and needs to be reinstalled.
patch-work
commented
1 month ago I haven't read the rest of the ticket, but your error message says "Encode.c: loadable library and perl binaries are mismatched", so the
Encodemodule on your system was compiled for a different (incompatible) version of perl and needs to be reinstalled.
So it seems, according to perldiag:
%s: loadable library and perl binaries are mismatched (got %s handshake key %p, needed %p)
(P) A dynamic loading library ".so" or ".dll" was being loaded into the process that was built
against a different build of perl than the said library was compiled against. Reinstalling the
XS module will likely fix this error.However, I was able to solve the problem by removing collisions, namely uninstalling packages.
See https://github.com/fastmail/authentication_milter/issues/161 for a similar example, and related struggle.
bulk88
commented
1 month ago doas /usr/local/bin/spamd --help Encode.c: loadable library and perl binaries are mismatched (got first handshake key 0xeb80000, needed 0xf280000)
So, there is a conflict, but the error message does not help tracking down the root cause of it.
0xeb80 and 0xf280 are sizes of the interp struct/my_perl. The 2 numbers are so far apart, this isn't even a CCFLAGS -D dropout by Makefile.PL. I will guess you have 5.30.0 perl.bin competing with 5.40.0 perl.bin. .30 and .40 are randomly picked guesses. Just use strace and identify the abs path of the perl bin, and where that XS .so is on your disk. You are using 2 different package managers, or your package manager is pointed at 2 different major release OSes, or you have 2 different perls or a mismatched perl vs libperl.so
patch-work
commented
4 weeks ago doas /usr/local/bin/spamd --help Encode.c: loadable library and perl binaries are mismatched (got first handshake key 0xeb80000, needed 0xf280000)
So, there is a conflict, but the error message does not help tracking down the root cause of it.0xeb80 and 0xf280 are sizes of the interp struct/
my_perl. The 2 numbers are so far apart, this isn't even a CCFLAGS -D dropout byMakefile.PL. I will guess you have 5.30.0 perl.bin competing with 5.40.0 perl.bin. .30 and .40 are randomly picked guesses. Just use strace and identify the abs path of the perl bin, and where that XS .so is on your disk. You are using 2 different package managers, or your package manager is pointed at 2 different major release OSes, or you have 2 different perls or a mismatched perl vs libperl.so
There is a single perl.bin in the whole system. However, this is an updated OS, so perhaps there are old libraries lying around.
leonerd
commented
4 weeks ago As a general note: I find the end-user experience here very poor. As a core perl developer I can justabout guess what's happened, but a more casual end user would have almost no hope at all.
I would greatly prefer (for the sake of not confusing users) if the handshake and message had something to say about the Perl version. 99 times in 100, that'll remind people what is going on.
I.e. instead of the message:
XS.c: loadable library and perl binaries are mismatched (got handshake
key 0x7ec0080, needed 0x7f00080)We got
XS.c: loadable library and perl binaries are mismatched (perl version is 5.40.0, library wanted 5.30.0)To an end-user, that second message is much more useful and much more likely to trigger them into knowing what is going on and how to fix it.
khwilliamson
commented
4 weeks ago +1000 from me too
bulk88
commented
4 weeks ago As a general note: I find the end-user experience here very poor. As a core perl developer I can justabout guess what's happened, but a more casual end user would have almost no hope at all.
XS.c: loadable library and perl binaries are mismatched (perl version is 5.40.0, library wanted 5.30.0)To an end-user, that second message is much more useful and much more likely to trigger them into knowing what is going on and how to fix it.
The code to emit that is later on but very unlikely to be reaching b/c previous checks, and if "friendlyness" matters, how do you prevent the even worse message?
XS.c: loadable library and perl binaries are mismatched (perl version is 5.40.0, library wanted 5.40.0)OS vendor or some 3rd party packer changed 1 random -D or Configure option in perl or added their not-P5P known "security patch". Float/NV length for example. Interp struct size not agreeing is fatal, none of the offsets can be trusted between perl.bin and module.so if interp structs don't agree.
I never thought back when I added this feature, that Linux people would hit the error, and that my SEGVs were Win32-unique to me keeping 2 perl.bin libperl.so'es and misc build trash in my PATH.
usage:
Perl_xs_handshake(U32 key, void * v_my_perl, const char * file,
[U32 items, U32 ax], [char * api_version], [char * xs_version])Checking Perl API string, before check interp size, is API impossible, unless you remove info (module name/package) from the existing message and switch to "Foo.c", b/c module name comes as SV * on Perl stack which is unusable/UB memory for mismatch.
https://github.com/Perl/perl5/blob/f2cb455cf7a4a7415dde08eb0999afcf37cb6a4a/util.c#L5624
It is possible to check Perl API string before interp size, but 1% risky if the arg is explicitly missing by XS module's author's demand since char * api_version is optional but 99% of CPAN modules will be supplying that arg. IIRC back in 5.20 or 5.12 when I added the API there were module authors or OS vendors that were bypassing Perl API version on purpose, either "demanding" ABI compat 5.8-5.12-5.infinity from P5P (request perm denied by P5P/P5P policy), or their module's Makefile.PL/Makefile was badly broken and stripping CCFLAGS and all -D'es or hard coding perl/lib/CORE or that module refused to use Makefile.PL and just included a hand written Makefile. I think that all has been fixed and char * api_version could be checked before interp size but then 2 Perl_croak_nocontext("Perl API version %s of %" SVf " does not match %s", api_p, SVfARG(PL_stack_base[ax + 0]),"v" PERL_API_VERSION_STRING); statements exist, noperl_die( variety and Perl_croak_nocontext( variety. Maybe Perl_croak_nocontext( variety can be removed forever as it is almost unreachable without dieing on interp size mismatch first.
Note for future, SV * is UB for ABI mismatch, it used to be 3 PTRs big before sv_any was added. version obj is UB ABI also. SV * could be 5 PTRs big in future, or SV * could be a Firefox or Chrome ptr in the future. FF NaN boxing (all SV's are NaN floats, hardware float exception OS signal, triggers "SV magic"). Or Chrome's "real objects, lowest 2 bits are always zero", if low 2 bits are not zero, the value is IMMORTAL.
bulk88
commented
4 weeks ago Another question, are OS native missing symbol messages any better than our current binary mismatch X vs Y in Foo.c?
Signal SEGV received core dumped EOM is worse for missing symbol. or "symbol Perl_whatever failed to resolve" is worse, no file name.
should Perl_xs_handshake be Perl_xs_handshake_5_40_0() through a macro? Note this won't stop 2 libperls in 1 process problem, because 5.42 perl.bin can load 5.40 xs.so that loads 5.40 libperl.so and no missing symbol.
does Dynloader.xs have a ENV var to trace what .so is actually loaded? is the abs path known of the XS module on all platforms?
nrdvana
commented
3 weeks ago My suggestion on p5p list:
"Perl module $full_module_name tried loading shared C library $full_path_to_lib, but that library was compiled for a different perl than the current $full_path_to_interpreter. This is often caused by updating your installed perl without updating the modules to match, or a mis-configured library path. See perldoc $relevant_section."
tonycoz
commented
3 weeks ago does Dynloader.xs have a ENV var to trace what .so is actually loaded? is the abs path known of the XS module on all platforms?
Both Dynaloader and XSLoader do push the module name onto @DynaLoader::dl_modules before calling the boot_* function but from looking at Perl_xs_handshake() you appear to be avoiding using Perl APIs before a successful key check.
If we could use APIs we could fetch the last element of @dl_modules and use that in the message. Similarly the filename for the loaded module is pushed to @DynaLoader::dl_shared_objects.
XSLoader/DynaLoader could be modified to set some name in the environment, but environment access isn't generally thread-safe.
nrdvana
commented
3 weeks ago avoiding using Perl APIs before a successful key check
I didn't realize the checks were running from the code of the newly loaded module. That makes everything a lot harder. Could the failure of the new module to initialize pass back an error to the Dynaloader code so that the host perl can generate the error message, taking full advantage of the facilities to tell it what paths it was dealing with?
Or, is it possible for the host program to look up a string in the .so / .dll before the initialization runs? Then it could do a more intelligent analysis of the .so before running its initializer code.
tonycoz
commented
3 weeks ago I didn't realize the checks were running from the code of the newly loaded module. That makes everything a lot harder. Could the failure of the new module to initialize pass back an error to the Dynaloader code so that the host perl can generate the error message, taking full advantage of the facilities to tell it what paths it was dealing with?
No, the checks are done in the perl (or libperl.so) binary, xsubpp generates a call to the dXSBOOTARGSXSAPIVERCHK macro in the boot function of the loadable module, which ends up calling Perl_xs_handshake() in perl/libperl.
I suspect the current code doesn't use perl APIs before the signature check out of an abundance of caution, @bulk88 originally wrote that code so I was checking with them.
Or, is it possible for the host program to look up a string in the .so / .dll before the initialization runs? Then it could do a more intelligent analysis of the .so before running its initializer code.
I don't know how portably we could do data lookups.
We could (I suppose) add a signature_* entry in addition to the boot_* entry, and that signature entry could return the fields that matter here and the module name.
That would require xsubpp to be updated and existing binaries wouldn't have the entry.
bulk88
commented
3 weeks ago does Dynloader.xs have a ENV var to trace what .so is actually loaded? is the abs path known of the XS module on all platforms?
Both Dynaloader and XSLoader do push the module name onto
@DynaLoader::dl_modulesbefore calling theboot_*function but from looking at Perl_xs_handshake() you appear to be avoiding using Perl APIs before a successful key check.
The handshake code can only use very basic libc fn calls, and kill the process ASAP. You can't use SV *s/PerlIO/any Perl C APIs. There could be 2 different addresses PL_stack_sp global symbols in the proc if a second no-thread libperl (atleast in Win32 DLL symbol visibility world) automatically loaded into the proc's address space.
So the Perl stack @_, can NOT be used to pass anything through Perl_xs_handshake(), until my_perl ptrs/interp size/&PL_stack_sp ptrs are compared, and effectively it was verified that the Perl_pp_entersub() fn ptr and Perl_xs_handshake() fn ptr live in the same libperl .elf/.so./.exe/.dll file.
That Foo.c was the only safe, sane, and SEGV proof, piece of data, I saw, that Perl_xs_handshake(), can print to console. In a just-loaded libperl, where all C globals are uninited in that libperl, and my_perl/Perl_get_context() basically will SEGV near instant b/c interp structs don't line up, or new libperl's OS TLS is uninitted, and who knows the "age"/major_rel_ver of the incoming SV *, and the SV header flags are binary garbage. Its impossible to move that data over any current existing API.
You can't write a hypothetically, Devel::Peek, where ONE .so/XS module, no compiling, no collection of .so fles, ONE. .so can correctly decode SV *s from memory dump file, simultaneously from Perl 5.10-5.40. With no crashes. Challenge issued!
Perl_xs_handshake() must not crash, and basically be forever forwards and backwards compatible "enough", mixing random XS modules and random major vers of libperl. Thats how I designed it. DO NOT SEGV is the most important goal.
IMO perlwarn and Dynaloader:: absolutely need doc revision how to find the abs path of the offending XS shared-lib. and it needs to be more user friendly than "learn to use strace" or start up MS ProcMon (much easier GUI).
I think a shell ENV var to control dumping the .so paths to console, is needed. ``Dynaloader::``` should read the ENV var once (perf) on perl-proc startup, and save the flag as a bool in PP space or XS/C space, then dump all the paths to STDERR.
UI note, many sys admins/power users, can't read Perl script, don't know what a BEGIN {} is, or the "mismatch" is in some enterprise Perl server stack, perl is running in another OS user account/service mode, and and the user can't change perl.bin s command line args or doesn't know what the args are. And the user can't PP step-debug the app either. The "server" won't start on the shell (missing secrets/chroot/etc) and "mismatch" can't be reached interactively. Therefore, ENV var I suggest.
If we could use APIs we could fetch the last element of
@dl_modulesand use that in the message. Similarly the filename for the loaded module is pushed to@DynaLoader::dl_shared_objects.
Other than Dynaloader:: dumping debug info before the fatal handshake it wont work. Remember handshake is instant-fatal incase Perl_sv_setsv() from a 5.24 libperl was injected into libperl 5.40 on ELF with LD_PRELOAD.
I dont see a API way to get that PP Dynaloader metadata strings across. Long ago, designing Perl_xs_handshake(), a number of problems.
-the XSUB's CV * is the ONLY way to pass data to BOOT XSUB and then to Perl_xs_handshake(), the XSUB's C prototype can't be changed, well, maybe it can be, but that is adventerous!!!!
-xsubpp/::ParseXS is slow to cut major releases on CPAN/merge patches. P5P is faster than ::ParseXS's maint dev.
-must deal with older xsubpp versions and their generated code
-no portable/cross platform Perl API exists for a XS shared lib, to pass it's disk file name to Perl_xs_handshake().
Aka these 2 on POSIX, https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-getmodulefilenamea and https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-getmodulehandleexw GET_MODULE_HANDLE_EX_FLAG_FROM_ADDRESS (0x00000004).
We could (I suppose) add a signature* entry in addition to the boot* entry, and that signature entry could return the fields that matter here and the module name. That would require xsubpp to be updated and existing binaries wouldn't have the entry.
How POSIX/spec safe is it (__cdecl vs your OS and CPU), to mix non-vararg func ptrs with vararg func ptrs? Do extra/ignored C args, passed thru vararg fn ptr type cast, do those extra C args, crash existing vararg-unaware compiled machine code functions?. I know the answers on Win32, but what about ARM/SysV/SPARC/POWER/etc.
Can XS bootstrap:: be special cased, as a XSUB that takes 2 args, CV * cv and const char * abs_so_path (unwrapped SVPV), then pass arg 2 to xs_handshake() using xs_handshake()'s extendable design (vararg+master control word as arg 1). Now xs_handshake() in an uninit globals about to SEGV, newly loaded libperl, has a better string to print to libc terminal?
Maybe as described above, "bootmeta_Foo" data symbol is needed, which is a C global Data symbol export, from the XS .so to parent perl, and the ,SO's const struct has 3-part Perl API version string. No machine code has run yet from this contaminated .XS .so, so we are still in full control, on the master/root libperl, and the boot XSUB is never executed if the checks fail against the global exported symbol, RO const C struct, data table, if the module's C struct of meta data is 100% incompatible with this perl build. This would be trappable and continue life with no segv risk. Not fatal error we have currently, Simply unload the .so, (which will hopefully unload that 2nd pesky out of place libperl) from OS lib loader,, never running 1 CPU opcode from the .so, and life continues,
I think a debug mode/trace for Dynaloader is easier. Than forklifting XSUB to Gen2 XS. Somewhere on my list, I have plants/experiemtnts for perl core or xsubpp ppport.h help, of storing the `CV * cv = newXS(CV)`; wayy wayyy different than today. LETS GO PARSE/LOOP THE DLL's export table. Because Win64 DLLs use 32bit RVAs to fnc ptrs and 32bit or 16bit offsets to Cstring labels for symbol names on DLLs. NOT 64b ptrs with upper 5 bytes being always NULL. Putting XSUBs in teh DLL EXPORT table and have perl parse and import that, would be super more space efficient on disk on x64 machines, because in our current set up we can only use plain C ISO types and add machine code ops ontop of that which are space wasting on x64, and can't use OS ABI specific 16 bit or 32bit ptrs/offsets feature in 64b process to store 100s or 1000s of XSUB fnc ptrs, package name strings, more compactly. On some OSes, crash dump backtraces will look much more pretty, since OS Native tools use the export tables for make faux C backtraces with CC debugging symbols are unavail.
A formal design, on how to pack a XSUB's Perl name, and its protoype, signature, etc, basically C++ name manging for XSUBs, so that 1 Cstring in the linker, has all info for Perl to attach those XSUB fnc ptrs to PP land, has to be spec defined/community API designed.
Im going to rewrite the XS Handshake messages RN, for some UI improvement,
Always show API ver string mismatch error before interp size mismatch if both failed safety tests. API Ver string mismatch error is very auto-self-resolving, each user will be face with a choice, they has to force reinstall the complete perl from either the left side or right side perl version number. Using their OS package manger/vendor in --force or --repair mode or uninstall mode.
XSLoader/DynaLoader could be modified to set some name in the environment, but environment access isn't generally thread-safe.
Baby new libperl.so that was accidentally loaded, will SEGV trying to read env vars I think from XS mode. We hook most of libc tokens and route them to perl APIs, but in this case, we have an uninitted libperl.so running, So SEGV time if malloc is used or SV *s are used.
With alot of testing perhaps good libperl.so and bad new libperl.so, can communicate through env vars, with no SV areanas, aslong as its 100% pure lib C.
I'd rather VARARG bootstrap:: XSUB and get freakiy with ::ParseXS which is the heavy engineering fix, and add a 2nd const char * to var arg xs_handshake() with the abs path of the DLL, which was computed. from the good/safe/clean parent interp before the toxic .so was loaded into VM space.
Revision: 724cf8d623675450a9b7be794b6b3917972e83f8
Author: Nicholas Clark <nick@ccl4.org>
Date: 5/1/2011 5:12:15 PM
Message: Move PL_global_struct_size, PL_interp_size{,_5_16_0} to perl.h
Make them const U16 - they should have been const from the start.It seems NClark was working on somekind of solution to this XS .DLL vs Perl.exe version war, but he never finished it, and not enough code or commit comments to explain it. Maybe he can come by and give the API proposal/design proposal from way back then
Leont
commented
3 weeks ago -the XSUB's CV * is the ONLY way to pass data to BOOT XSUB and then to Perl_xs_handshake(), the XSUB's C prototype can't be changed
Yeah, in hindsight it's obvious a different set of arguments would have been helpful.
I can kind of imagine we'd add some bootstrap2 symbol with better semantics, and then DynaLoader uses that from now on. There is some wiggle room if we make our moves right.
How POSIX/spec safe is it (__cdecl vs your OS and CPU), to mix non-vararg func ptrs with vararg func ptrs?
I think that would be probematic on recent Mac OS, but I don't quite know the details.
nrdvana
commented
3 weeks ago How about this? (rough draft, for discussion)
Built in each XS during compilation:
extern uint32_t xs_signature= 0x12345678;
extern const char xs_metadata[]=
"perl_version\0" "5.42.0\0"
...
...
;
extern uint32_t metadata_My__Module(const char **meta_out, uint32_t *meta_len_out) {
if (meta_out) *meta_out= xs_metadata;
if (meta_len_out) *meta_len_out= (uint32_t) sizeof(xs_metadata);
return xs_signature;
}Then this in perl:
diff --git a/ext/DynaLoader/DynaLoader_pm.PL b/ext/DynaLoader/DynaLoader_pm.PL
index 0f13f9aae0..88486846e4 100644
--- a/ext/DynaLoader/DynaLoader_pm.PL
+++ b/ext/DynaLoader/DynaLoader_pm.PL
@@ -415,6 +415,18 @@ sub bootstrap {
Carp::carp("Undefined symbols present after loading $file: @unresolved\n");
}
<</$^O-eq-freemint>>
+ my $metaname = ("metadata_$module" =~ s/\W/_/gr);
+ my $meta_ref = dl_find_symbol($libref, $metaname);
+ if ($meta_ref && dl_read_xs_metadata($libref, $meta_ref) != $perl_xs_signature) {
+ my %meta= dl_read_xs_metadata($libref, $meta_ref);
+ croak("Perl module $module tried loading shared C library $file, but that library "
+ . ($meta{perl_version} ne $]
+ ? "was compiled for perl version $meta{perl_version} rather than the current perl $] at $full_path_to_interpreter. "
+ : "was compiled for a different perl than the current $full_path_to_interpreter. "
+ )
+ . "This is often caused by updating your installed perl without updating the "
+ . "modules to match, or a mis-configured library path. "
+ . (defined $meta{distro_reinstall_command}? "Try running $meta{distro_reinstall_command}"
+ : defined $meta{distro_package_name}? "Try reinstalling $meta{distro_package_name}"
+ : "See perldoc $relevant_section."
+ );
+ }
$boot_symbol_ref = dl_find_symbol($libref, $bootname) or
croak("Can't find '$bootname' symbol in $file\n");
diff --git a/ext/DynaLoader/dl_dlopen.xs b/ext/DynaLoader/dl_dlopen.xs
index c3c874e027..bdb44250bc 100644
--- a/ext/DynaLoader/dl_dlopen.xs
+++ b/ext/DynaLoader/dl_dlopen.xs
@@ -274,6 +274,34 @@ dl_install_xsub(perl_name, symref, filename="$Package")
filename, NULL,
XS_DYNAMIC_FILENAME)));
+# Called in scalar context, it returns the XS signature.
+# Called in list context, it returns key/value pairs.
+
+void
+dl_read_xs_metadata(libhandle, symbolref)
+ void * libhandle
+ void * symbolref
+ INIT:
+ uint32_t (*meta_fn)(const char**,uint32_t*)= (uint32_t)(*)(const char**,uint32_t*) symbolref;
+ uint32_t signature, meta_len;
+ const char *meta, *start, *pos, *lim;
+ PPCODE:
+ if (GIMME_V == G_LIST) {
+ signature = meta_fn(&meta, &meta_len);
+ PUSHs(sv_2mortal(newSVpvs("xs_signature")));
+ PUSHs(sv_2mortal(newSVuv(signature)));
+ for (pos= start= meta, lim= meta + meta_len; pos < lim; pos++) {
+ if (!*pos) {
+ PUSHs(sv_2mortal(newSVpvn(start, pos - start)));
+ start= pos+1;
+ }
+ }
+ if (pos > start)
+ PUSHs(sv_2mortal(newSVpvn(start, pos - start)));
+ } else {
+ signature = meta_fn(NULL, NULL);
+ PUSHs(sv_2mortal(newSVuv(signature)));
+ }
SV *
dl_error()The advantage of the above is that now you can also write a tool that scans a directory of perl .so files and pulls the metadata from each of them, for e.g. writing a script to detect outdated modules. Could be very useful for distro maintainers, especially if one of the strings we provide in that metadata is the distro's package name for the module. That could be facilitated by the distro setting an environment variable during compile so that the XS module knows what distro package it belongs to, and could be done without any cooperation by the XS author.
tonycoz
commented
3 weeks ago The handshake code can only use very basic libc fn calls, and kill the process ASAP. You can't use SV *s/PerlIO/any Perl C APIs. There could be 2 different addresses
PL_stack_spglobal symbols in the proc if a second no-thread libperl (atleast in Win32 DLL symbol visibility world) automatically loaded into the proc's address space.So the Perl stack
@_, can NOT be used to pass anything through Perl_xs_handshake(), untilmy_perlptrs/interp size/&PL_stack_spptrs are compared, and effectively it was verified that the Perl_pp_entersub() fn ptr and Perl_xs_handshake() fn ptr live in the same libperl .elf/.so./.exe/.dll file.
Thanks, that had slipped my mind. On Linux at least, the loadable objects don't seem to reference the perl executable or libperl.so, for example, for the packaged Imager.so for my system perl, which isn't -Duseshrplib:
$ ldd /usr/lib/x86_64-linux-gnu/perl5/5.36/auto/Imager/Imager.so
linux-vdso.so.1 (0x00007ffdcc1bc000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f090ad54000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f090ab73000)
/lib64/ld-linux-x86-64.so.2 (0x00007f090aeed000)If I build a -Duseshrplib perl, similarly there's no reference to libperl.so:
$ ldd lib/auto/GDBM_File/GDBM_File.so
linux-vdso.so.1 (0x00007ffe49f90000)
libgdbm.so.6 => /usr/lib/x86_64-linux-gnu/libgdbm.so.6 (0x00007f4ebd896000)
libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007f4ebd6b5000)
/lib64/ld-linux-x86-64.so.2 (0x00007f4ebd8b9000)so they won't load a foreign libperl as Windows might.
As to the choice between the loadable object reporting metadata that DynaLoader checks and an extended xs_handshake, I think I prefer the metadata, since it means we're not giving control to a possibly mismatched bootstrap.
Ideally we'd fetch the data pointer directly with dlsym()/GetProcAddress() since it avoids passing control to the loadable object but I'm worried about portability for name decoration (_ prefixed names on 32-bit Windows for example).
I expect the exported metadata can be more extensible than a re-worked xs_handshake too.
tonycoz
commented
3 weeks ago One problem I do see is loadable modules that are hand-written or are generated without xsubpp (such as with SWIG) won't have the metadata. SWIG's generated boot_* function doesn't appear to include the handshake call.
patch-work
commented
3 weeks ago When the new machinery will report a more informative error, please make sure it will also report the files being checked. It is of little use to learn of a mismatch in perl versions when you are still blindfolded and need to find the relevant file to fix.
nrdvana
commented
3 weeks ago One problem I do see is loadable modules that are hand-written or are generated without xsubpp (such as with SWIG) won't have the metadata. SWIG's generated
boot_*function doesn't appear to include the handshake call.
In my example, I started with if ($meta_ref &&, so it's not required. You just won't get a nice error if the metadata isn't present.
nrdvana
commented
3 weeks ago Ideally we'd fetch the data pointer directly with dlsym()/GetProcAddress() since it avoids passing control to the loadable object but I'm worried about portability for name decoration (_ prefixed names on 32-bit Windows for example).
I was thinking the same initially, but my suggested getter should be about as safe as you can get. If the .so isn't the right architecture it won't load in the first place, and it does nothing more than copy values into pointers.
bulk88
commented
3 weeks ago The handshake code can only use very basic libc fn calls, and kill the process ASAP. You can't use SV *s/PerlIO/any Perl C APIs. There could be 2 different addresses
PL_stack_spglobal symbols in the proc if a second no-thread libperl (atleast in Win32 DLL symbol visibility world) automatically loaded into the proc's address space. So the Perl stack@_, can NOT be used to pass anything through Perl_xs_handshake(), untilmy_perlptrs/interp size/&PL_stack_spptrs are compared, and effectively it was verified that the Perl_pp_entersub() fn ptr and Perl_xs_handshake() fn ptr live in the same libperl .elf/.so./.exe/.dll file.Thanks, that had slipped my mind. On Linux at least, the loadable objects don't seem to reference the perl executable or libperl.so, for example, for the packaged Imager.so for my system perl, which isn't
-Duseshrplib:$ ldd /usr/lib/x86_64-linux-gnu/perl5/5.36/auto/Imager/Imager.so linux-vdso.so.1 (0x00007ffdcc1bc000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f090ad54000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f090ab73000) /lib64/ld-linux-x86-64.so.2 (0x00007f090aeed000)If I build a
-Duseshrplibperl, similarly there's no reference tolibperl.so:$ ldd lib/auto/GDBM_File/GDBM_File.so linux-vdso.so.1 (0x00007ffe49f90000) libgdbm.so.6 => /usr/lib/x86_64-linux-gnu/libgdbm.so.6 (0x00007f4ebd896000) libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007f4ebd6b5000) /lib64/ld-linux-x86-64.so.2 (0x00007f4ebd8b9000)so they won't load a foreign libperl as Windows might.
As to the choice between the loadable object reporting metadata that DynaLoader checks and an extended xs_handshake, I think I prefer the metadata, since it means we're not giving control to a possibly mismatched bootstrap.
Ideally we'd fetch the data pointer directly with dlsym()/GetProcAddress() since it avoids passing control to the loadable object but I'm worried about portability for name decoration (
_prefixed names on 32-bit Windows for example).I expect the exported metadata can be more extensible than a re-worked xs_handshake too.
Remember prior insanity P5P dealt with. Win 16, 32b WinCE, Symbian, "shared libraries", yes the ".dll", is a service daemon that lives longer than the process. Yes, the contents of C global vars in the .dll, live longer than the C global vars of the .exe and live longer than the Process ID. Perl.bin and perl PIDs come and go and C globals in perl.bin cone and go and zero fill the normal unix way, but the C global vars in libperl.dll, last forever until the "power button" is used.
The only way to wipe zero fill all your C globals, choice 1, keep them in 1 struct contiguous, choice 2, array of void *s to C globals to zero fill. Choice 3, to zero fill your C global vars, reset button, power switch, or unplug power cord from wall.
Also, I think P5P supported OSes, where if the perl process, exec() open() the perl.bin disk file again, the OS just calls C "main()" in a new thread in the same process/virtual address space. You can't clear/zero fill the perl C global vars in perl.bin until all perl PIDs exit.
IIRC, this is now the weird Perl build configuration of having my_perl pointers args but no ithreads:: and no pseudofork:: and #ifdef away the libperl functions that are used for CGI/apache/nginix/Varnish.
Or a weird config of no thread perl keeping " interp struct" as C globals in a tiny disk size perl.bin. with a fat 2-4MB libperl where the real code of The Interpreter lives.
Now this is drinks talk but remember that on iPhone and Android the design architecture of a process is identical, to the MSDOS implementation of the process . Both Android and Iphone, are single threaded single process operating systems. Note, my Samsung A42 from 2020, was really single thread single process but the Samsung a54 from 2023 I'm holding right now is technically multi-processing and I can't find a good use of split screen at all on a smartphone PIP is great but split screen is useless. And it's too many swipes to turn off split screen when I accidentally turn it on.
I got the idea yesterday, the number one complaint is strace is the only diagnostic tool. And the top four most wanted criminals running from the law, have the street names and pseudonyms of "XS.c", "Class.c", "Base.c", and "Parent.c". If you recognize any of these four felons immediately call the police.
My bin analyzer shows "Foo.c" has only one callar which is bootstrap xsub, which xs_handshake() and newXS() actually mandate. Those 2 mandates chain to macro/struct field CvFILE(). What is CvFILE()? Do not copy paste anything from Perl API Pod Perl intern Pod or Perl XS Pod or the .h files in your answer.
bulk88
commented
2 weeks ago When I actually had working Windows CE Pearl 32b port, fun fact you learn very quickly using a live C debugger on Windows Mobile 32b operating system. The hard drive, the free space count, on the hard drive, the concept of a file, and the concept of a directory, are artificial concepts, that are invented and only exist in, inside Microsoft Libc library compiled for Windows Mobile OS. If you use a raw sector hard drive GUI tool on Windows mobile OS. And hunt down perl.bin the file, you'll realize the absolute position, in that hard drive tool, on disc, of perl.bin, bizarrely is the same integer, as C ptr addr, showing in your C debugger, of a paused Perl process.
You know where this is going.
Windows mobile OS, in the kernel SDK, and in a GUI device manager for the phone, you're not going to find a single driver, object instance, C structure, called "flash disk controller" or "hard drive". Those phones use "static RAM" soldered next to the volatile RAM on the PCB. That OS and those phones do not use SSD flash chips which have sector addresses, and do not have asynchronous IO concept of queuing a read request and then waiting for a CPU interrupt. In the 1990s and 2000s doing this Unix nonsense of a "stream" "file descriptor" allocating and deallocating malloc blocks representing IO requests to a pointless "hard drive controller" was an unacceptable use of battery life CPU time and silicon. Nobody user doing anything on these phones will actually ever notice GUI wise, with their finger, that the "hard drive" is synchronous blocking I/O, on the ram bus of the CPU. The CPU just freezes for microseconds or nanoseconds while The cache Line strobes in on the PCB to the CPU.
Even more fun trying to run Perl for Windows CE port, all your developer tricks from hacking Windows 98 and NT-11, from your Desktop don't work on Windows CE. So naturally it's time for me to go attack "kernel32.dll", right? WRONG. WinCE FS is "/" and "/Windows/" and "/Storage Card/". The file browser says "/Windows/System32/kernel32.dll" is a 3KB file, weird but whatever I copy it to my desktop. I open WinCE K32.dll in a Desktop PE tool, these are basic windows programming tools. It reported there is no executable code inside kernel32.dll and all of the fn pointers of exported C symbols are invalid memory addresses pointing outside of the dll.
After a lot more research I learned that all the dll files you see in the file browser on Windows CE are only there to keep Desktop Windows developers happy and win 32 API programs happy and to keep end users happy so an end user does not freak out when they buy their new phone at the store. It was assumed the user will freak out and run back to the store when they realize on their new phone the one and only file they can find with the file picker. Is an empty "/home" directory.
So the entire Windows mobile phone is a fat packed single compiled C binary .exe produced by visual c. There is no filing system basically. Every entry in the "program files" directory and "windows directory" is faked by "MS libc for WinCE" to keep developers and users happy. All win32 devs got tricked by libc thinking the phone is full of apps and files when there are no apps there are no files at all and the all end users got tricked by the file picker gui. Those fake files, are implemented by RO C global array of structures. Each struct is void to C symbol/var of the .jpeg/.txt/.wav/.exe, and char to file name. In another place in libc, in RW Global memory, there is a bit mask of soft delete for those fake files. Once that bit is flipped the fake file is gone from libc forever unless you do a factory reset. And because long tap delete makes the icon disappear in the file picker everyone has been tricked. This keeps win32 API Developers and phone users happy enough and stops the complaints. Nobody picks up on the tiny detail that deleting Microsoft bloat in the windows folder does not change the free hard drive space reported in the file picker at all. They also might notice that if they delete a .jpeg a .gif or a .mpeg, then upload another jpeg to the phone and put it in the same directory as the first jpeg they deleted. The user is trying to change the splash screen they call ugly in a Microsoft Factory app in the phone and looking with the file picker they realized that the ugly splash screen background happens to be in program files as a jpeg file and then thought they will change the ugly logo in the app by changing that file with the file picker with a new jpeg. This is guaranteed to change the splash screen on desktop Windows 32 and it will change the splash screen on a side loaded Windows CE app but it's not going to change the ugly splash screen in a Microsoft Factory app or your cell network branded ugly messaging app that came with the phone (Verizon USA). That jpeg is only in program files for psychological reasons, the app is actually C link time permanently bound to the jpeg. Stdio is never used to read or write that jpeg on the fake file system. App is wrong word, there are no apps on the phone it's a single.exe they are C function calls combined with the c-linker into one static link.
So Mr power user to fight the Microsoft Monopoly wants to delete the bloated "Internet Explorer Mobile 6" from his Windows Mobile phone and install Netscape. They go into program files, long tap, delete "iexplore.exe", go into Windows, system 32, long tap, delete "mshtml.dll". This is where things start to go wrong compared to desktop Windows. Mr power user recovered 0 bytes of hard drive space by doing this. MS Outlook for Mobile keeps showing HTML emails like nothing happened. And probably half of your own personal side loaded apps will not start up anymore, with error box about missing dll error.
So that's a very interesting operating system that I had a chance to work on and they were very few complaints and almost no developers or end users ever noticed this cuz if they use official Microsoft APIs they would have never figured this out. Very important detail Perl for window CE 32b and all Side loaded personal apps you install, the disk formats, the runtime API begavior, the C struct names and field layouts, the C function symbol names, all of this will be identical to 32b desktop Windows. The most shocking glaring difference is only finding arm 32 machine code instead of Intel x86 lying around in virtual memory and in the disk files , nothing else changed for a side load app, developer experience, C compiled time, and user runtime wise. 90% of fns from desktop Windows 95 kernel32.Dll are available on Windows CE, the last 10% such as current Drive letter are missing symbols and a porting headache.
The fat packed fake .exe and fake .dlls problem is only for apps that are factory installed on your Windows mobile phone. Sideload is normal disk files like any OS. Deep secret from Windows CE OS, there is no msvcrt.dll, all lowercase C functions are exported by kernel32.dll. that is not the business of an app developer or an end user to know that.
I did ask on a forum how do I hack/patch/replace kernel32.dll on my phone? It's a 3KB file.
The answer I got is not pretty.
"Syscall VirtualProtect(ptr, PROTECT_RW) does not work if the Addr is 0x60000000-0x80000000, that is the ROM chip, in your phone and that is your 90 MB fat pack OS image. Programming voltage is 7 Volts for 5 minutes to erase the memory cells on that thing. You need a new full OS image available once you have completed the erasing procedure. You cannot change kernel32.dll any other way. You are going to need a pirated copy of "Oak SDK", obviously you're already are a c-developer but you're going to have to run "nmake" and compile the entire OS from .c, although that's the easy part, now this is the hard part you cannot just compile, a pirated copy of Windows mobile, where are you getting the Qualcomm drivers for your phone from? and you already know you can't copy the files from the file picker cuz the drivers .sys are 1 KB big. So here are some links to some warez cracking tools, that hopefully will create real .sys driver files that don't brick your phone, and don't instantly SEGV themselves. I hope you understand the implied extreme difficulty of what you are trying to do and there is no software tool that can perfectly automate recreating the Qualcomm drivers for the SOC on your phone. I suggest you find a new hobby bulk88 and don't try this."
So, I'm going to destroy my phone, and a 2nd one, or live not have a cellphone, while single stepping in arm32 machine code, with visual C debugger, for 2 weeks, or maybe a month of time, what pieces of arm machine code were missed, and that are scattered around by the visual c-linker, this closed Source pre GitHub semi-legal tool, the pieces of machine code this tool missed, when trying to recreate standalone win32/window CE kernel drivers .sys files, from a single compiled win32/wince binary. I agree, they are better ways to volunteer time.
Sometimes Samsung or HTC or the other vendors of Windows mobile phones made mistakes in the build process and fed an entire .sys file to the makefile that builds the operating system, instead of a .o file, to the OAK sdk makefile. That's an error by someone at Samsung or HTC. The file is a real PE file now sitting in a c array Char8_t array. If the phone vendor makes this mistake and includes the real driver that let hobbyists compile the OS and have a jailbroken Windows mobile phone running a custom ROM but my particular model that mistake was not made and you can't copy paste the drivers out of the phone. Also for some HTC phones HTC made an upgrade install Shield. EXE to upgrade some app on the phone and HTC accidentally included enough new dlls or accidentally included some updates to a factory driver and somehow they included a real full blown PE header driver in that install Shield package and now a hobbyist can recompile the entire OS and change anything they want but those were accidents or sloppy build Scripts and my model did not have that.
This is a very educational lesson on how not to bake architectural differences accidentally into software.
So does anyone know if Perl 5.40 still works on 286 segmented memory? Remember pointers always were 32 bits on 16-bit processors. You can't reach 640KB of RAM with 16-bit pointers.
Also not obvious architecture feature, C89/C99/C23, var arg feature, I hope all developers know that "va_start();" and "va_arg();" macros are actually ring 0 kernel syscalls, outside user mode virtual address space? They are ring 0 kernel system calls from the viewpoint of the ISO C spec.
The original reason was to bank switch 65 KB prefixes in segmented memory on your Intel 8086 or 286. The more modern reason is because Solaris on SPARC and Win64 on Itanium supposedly have unbreakable Hardware protection against against all C stack buffer overflows. Call/return assembler commands on SPARC CPUs pop and push the current executing instruction pointer into kernel space memory. Zero overhead, it executes in a fraction of a nanosecond, it's implemented in Hardware. C "longjmp()" either has a SPARC assembler op "rewind_call_frames(size_t layers)" or help from a kernel syscall. The integer addresses of those SPARC call frames are always a secret from user mode even if you can rewind them whenever you want. You can rewind them however much you want until SEGV but you're not allowed to know the integer addresses ahead of time before you issue the rewind op(). So don't mess around/hack/modify, "struct jmp_buf", you really don't want to know what's in that struct, or stick to Intel x86.
IA64 took this to the next level, from a document that I read a month ago for IA 64 assembly for Windows Server 2003. I did not have an IA 64 machine so this is just hobby reading. I want to say I was reading a tutorial for IA 64 assembly but this was not a guide for writing IA64 assembly this was a guide for writing IA64 assembly as hexadecimal bytes on paper and pen .The chance of causing C stack corruption, or a C stack security exploit on IA64 in the C language on Server 2003, is the same chance that you have of causing JavaScript stack corruption in Chrome from a .js or Java stack corruption on android from a .class. ZERO. IMPOSSIBLE. On IA64.
IA64 successfully converted C99 to use JavaScript memory protection model.
So from what I read on Windows Server for IA64, the .exe is always loaded at 0x10000000. Kernel32.dll is loaded at 0x10000000, every DLL in the process is loaded at 0x10000000. So how is that possible? Well Windows for Itanium uses the SysV ELF PLT/GOT, no you are not going to start that binary on Linux for IA64, it's still a PE file header and all the PE flags and PE structs just a little different. First major change, the C stack is fully separated between each C function call, just the way it is fully separated in JavaScript. IA64 assembly, call() op has a mandatory input argument of how much C stack bytes you want to roll back into Kernel space. C auto variables are unreachable between call frames in C. "= &sum_c_auto;" , is implemented in Hardware with an extra C var arg argument, the callee did not know the callee has. Also the return addresses are hidden in kernel space and are secret like on SPARC. The caller fn has full control with hardware help which C Auto variables become visible to the callee and which C auto variables are private hardware protected.
A single, 1 wrong extra "va_arg();" is guaranteed to segv on IA 64. Or your accidentally reading the memory address located passed as a size_t * 2 or 4 or 5 arguments to the left in your fn prototype. Which is perfectly safe to do you didn't learn anything new.
Yes so the C99 stack has been converted into the JavaScript stack.
Next design feature this was quietly killed by Microsoft and died with Server 2008 and died with IA 64. The PLT GOT was removed for Windows 8-11 for ARM.
All dlls and the exe, passed control of instruction execution pointer between dll exe boundaries through a wormhole fn through the PLT GOT. Each time you switch dlls or exes through the PLT GOT, map and remap the dll and exe at the exact same 0x10000000 hardware assisted. ~%10 of each dll and exe stays in place at the same address for the rest of the process runtime life, and 90% disappears and reappears each time you cross a dll or exe boundary. " = &static_decl_c_global;" in C99, causes the c-linker, sarcasm ahead to explain to non-technical people.
The C linker, make a paper sign that says "free car, steal me", drives to a high crime neighborhood, turns the car off leaves the keys on the driver seat, and walks away and will never see their car again.
The C linker put that C Global variable, in a COFF/PE section, at a 4096 sized memory page at a virtual memory address that is frozen forever for the rest of the runtime process life and is read writable no matter what DLL EXE is mapped at 0x10000000. Malloc memory is also Frozen at a fixed address forever ( until "mprotect()" or libc free() call, normal posix), and malloc memory visible to all dlls and the .exe, not mattering what is mapped in at 0x1000,0000.
A malicious root kit .dll, in a IA64 Win64 process cannot steal a private key stored in read only const C global memory, of another .dll, in the same process. That malicious rootkit .dll also cannot read as a Char buffer the machine code bytes that make up that other Dll. If the malicious.dll tries to do a memory address read it's just going to attack itself and its own memory mapped disk file and learn nothing new. This design stops all the basic easy security exploits but because all malloc memory is a swimming pool of everyone's (every dll) bacteria, you (malicious dll) could probably find some way to trick openSSL to accidentally copy the private key into a malloc buffer, by manipulating some C struct that happens to be in malloc memory, and exfiltrate the private key and send it to the hacker human.
This design was dropped by Microsoft and killed along with itanium, because number one it renders manually parsing the PE header useless because the pointers/0 indexed offets/RVAs that are in the PE header, if you read the PE header, are invalid at all times. Also the only PE header you can ever read is your own .dll PE header, because there can never be two images loaded at any time in the process simultaneously. Each thread and each CPU core is keeping another .dll mapped at 0x1000,0000 while that thread has CPU time and is using electricity. The flips are instant and done in Hardware. Microsoft abandoned this because there were developer complaints "GetProcAddress()" is doing runtime JIT and accused of generating RPC stubs, that are then "making JSON packets" and "using a socket", instead of "GetProcAddress()" obtaining the memory address of a c symbol fn ptr. It's not RPC it's not a socket. There is nothing on msdn api docs, and there is nothing in ISO C that makes this invalid. But the conversation ends with "Microsoft?do you like money? HP, do you like money? do you want money?" The response is "what about security?". Next answer, " the CEO has decided that next year's server purchase is Dell Xeons running Ubuntu click".
The zombie corpse of that architecture came back as https://learn.microsoft.com/en-us/windows/win32/secbp/control-flow-guard AMD and Intel did add one or two instructions to x64 because it is Hardware assisted with some pretty crazy but fast enough fallbacks in pure software for older iCore processors, specifically that all .dll and .exe are separated by a contiguous 4GB of invalid memory address space, and AMD x64 assembly only has a command called jump(const signed int32_t) or jump(register signed int64_t), static machine code analysis by the kernel or crypto signing the PE file and static machine code analysis by the c-linker, looking for jump register64, ops, starts to smell like virtual memory process isolation, but not the extreme of swapping All binaries and shared libs In and Out and keeping all of them at the same memory address.
The layer of security added on titanium is just useless ultimately because malloc memory is a lot more interesting to an exploit then read only constants in a dll, which can be exploited with normal stdio and a normal read(). Hiding the dlls and they fail file test operator/stat(), couldn't be done. And attempting to keep a private key or password completely as a C Auto in some C call frame sounds impossible the entire process run time. If you do &sum_c_auto, the hardware protection is broken.
https://learn.microsoft.com/en-us/cpp/build/exception-handling-x64?view=msvc-170
On win64 for AMD 64 day 1 this was also borrowed from IA64 Windows, but the actual C autos becoming segvs feature is missing. Notice all the information to make C Autos disappear between call frames is actually in that API.
bulk88
commented
2 weeks ago Answer for what is CvFILE(). It is dead code!! IE mathom code by P5P, from 1999. Somewhere a few years ago, someone proposed, why is "XS.c" and not "GTK/Build/More/Pain/XS.c"? Answer is technical debt/code rot.
This field had no usage in P5P perl except Devel::Peek. that is not usage. also 1 statement in builtin.c to fix in a couple seconds.
I can't find ANY non PRINT-STDERR use on CPAN for CV field CvFILE() except maybe, from reading the code, reini urban a decade ago, took over the CvFILE char * struct member as his personal property in B::C and B::CC and sticks more in the string, or more than just a file name string eq to CC's FILE token, in CvFILE, then references it later for internal purpose. He noticed it was dead code, and p5p isnt smart enough to notice the CvFILE() field themselves.
See
1999 https://github.com/Perl/perl5/commit/57843af05bc7863df9b9bfb6b37e3a29d08532a9
1999 labeled as dead code https://github.com/Perl/perl5/commit/b195d4879f55e1610299cb9b1b55356940c2a577
The introduction of field GvFILE()/GP struct/gp_file_hek, means, CvFILE() is absolutely dead and obsolete.
So, as someone proposed, shouldn't ParseXS just add Some/Module/" __FILE__ to the BOOTCHECK macros, or ParseXS/PPPort does this change and backports it, or P5P does this change? my bin analyzer shows, all the "Foo.c" strings in p5p core/libperl and in XS .DLLs, those Foo.c strings have zero reuse other than newXS() and xs_handshake(). Adding a few more chars to the string isn't that much bloat.
We might also have a Perl for Linux only problem, or Clang for Linux only or MSVC only problem, my strawberry perl has FULL PACKAGE NAMES like "lib\\File\\Map.c" and "lib\\Win32\\ChangeNotify.c" in handshake die() and in CvFILE(), IDK why ParseXS or GCC is putting full modules names in CC __FILE__. Dont have time to reserach it atm.
Migrated from rt.perl.org#130718 (status was 'open')
Searchable as RT130718$