search

Perl / perl5

🐪 The Perl programming language
https://dev.perl.org/perl5/
Other
1.9k stars 542 forks source link

Failed 3 tests out of 2552: ../ext/GDBM_File/t/fatal.t and ../ext/GDBM_File/t/gdbm.t #18466

Closed noloader closed 3 years ago

noloader commented 3 years ago

Description I'm working on Solaris 11.3, i86pc. I'm trying to build Perl 5.32 from sources.

The build is failing with:

Failed 3 tests out of 2552, 99.88% okay.
        ../ext/GDBM_File/t/fatal.t
        ../ext/GDBM_File/t/gdbm.t
        porting/exec-bit.t

Steps to Reproduce I wish I could provide a one-liner to reproduce the issue.

I built and installed all of Perl's dependencies, including an updated GDBM. Then I built Perl with the following command:

# Strip duplicate, leading and trailing colons
LD_LIBRARY_PATH=$(echo "$INSTX_RPATH:$LD_LIBRARY_PATH" | tr -s ':' | sed -e 's/^:\(.*\)/\1/' | sed -e 's/:$//g')
export LD_LIBRARY_PATH

# Use our up-to-date and hardened libraries.
# See http://www.linuxfromscratch.org/lfs/view/development/chapter08/perl.html
export BUILD_ZLIB=0 BUILD_BZIP2=0

    CC="${CC}" \
    CXX="${CXX}" \
    PKGCONFIG="${INSTX_PKGCONFIG}" \
    CPPFLAGS="${INSTX_CPPFLAGS}" \
    ASFLAGS="${INSTX_ASFLAGS}" \
    CFLAGS="${INSTX_CFLAGS}" \
    CXXFLAGS="${INSTX_CXXFLAGS}" \
    LDFLAGS="${INSTX_LDFLAGS}" \
    LDLIBS="${INSTX_LDLIBS}" \
    LIBS="${INSTX_LDLIBS}" \
./Configure -des \
    -Dprefix="${INSTX_PREFIX}" \
    -Dlibdir="${INSTX_LIBDIR}" \
    -Dpkgconfig="${INSTX_PKGCONFIG}" \
    -Dcc="${CC}" \
    -Dcxx="${CXX}" \
    -Acppflags="${INSTX_CPPFLAGS}" \
    -Aasflags="${INSTX_ASFLAGS}" \
    -Accflags="${INSTX_CFLAGS INSTX_CPPFLAGS}" \
    -Acxxflags="${INSTX_CXXFLAGS INSTX_CPPFLAGS}" \
    -Aldflags="${INSTX_LDFLAGS}" \
    -Aldlibs="${INSTX_LDLIBS}" \
    -Duseshrplib \
    -Dusethreads \
    -Dextras="FindBin Text::Template Test::More HTTP::Daemon HTTP::Request"

The INSTX_* variables are as follows:

  INSTX_BITNESS: 64-bits
   INSTX_PREFIX: /export/home/jwalton/tmp/ok2delete
   INSTX_LIBDIR: /export/home/jwalton/tmp/ok2delete/lib
    INSTX_OPATH: '$ORIGIN/../lib'
    INSTX_RPATH: /export/home/jwalton/tmp/ok2delete/lib

 AUTOCONF_BUILD: x86_64-pc-solaris2.11
PKG_CONFIG_PATH: /export/home/jwalton/tmp/ok2delete/lib/pkgconfig
       CPPFLAGS: -I/export/home/jwalton/tmp/ok2delete/include -DNDEBUG
        ASFLAGS: -Wa,--noexecstack
         CFLAGS: -g2 -O2 -m64 -march=native -fPIC -pthread
       CXXFLAGS: -g2 -O2 -m64 -march=native -fPIC -pthread
        LDFLAGS: -L/export/home/jwalton/tmp/ok2delete/lib -m64 -Wl,-R,'$ORIGIN/../lib' -Wl,-R,/export/home/jwalton/tmp/ok2delete/lib -Wl,-z,now -Wl,-z,origin
         LDLIBS: -ldl -lpthread
   CC: /bin/gcc
  CXX: /bin/g++

I don't see any log files to provide for you:

perl-5.32.0$ find . -name '*.log'
perl-5.32.0$

One thing I noticed... GDBM does not provide a pkgconfig (*.pc) file.

Expected behavior My fingers were crossed the build would succeed.

Perl configuration

$ LD_LIBRARY_PATH=".:./lib" PERL5LIB=./lib ./perl -V
Summary of my perl5 (revision 5 version 32 subversion 0) configuration:

  Platform:
    osname=solaris
    osvers=2.11
    archname=i86pc-solaris-thread-multi-64
    uname='sunos solaris 5.11 11.3 i86pc i386 i86pc solaris '
    config_args='-des -Dprefix=/export/home/jwalton/tmp/ok2delete -Dlibdir=/export/home/jwalton/tmp/ok2delete/lib -Dpkgconfig=/export/home/jwalton/tmp/ok2delete/lib/pkgconfig -Dcc=/bin/gcc -Dcxx=/bin/g++ -Acppflags=-I/export/home/jwalton/tmp/ok2delete/include -DNDEBUG -Aasflags=-Wa,--noexecstack -Accflags=-g2 -O2 -m64 -march=native -fPIC -pthread -Acxxflags=-g2 -O2 -m64 -march=native -fPIC -pthread -Aldflags=-L/export/home/jwalton/tmp/ok2delete/lib -m64 -Wl,-R,'$ORIGIN/../lib' -Wl,-R,/export/home/jwalton/tmp/ok2delete/lib -Wl,-z,now -Wl,-z,origin -Aldlibs=-ldl -lpthread -Duseshrplib -Dusethreads -Dextras=FindBin Text::Template Test::More HTTP::Daemon HTTP::Request'
    hint=recommended
    useposix=true
    d_sigaction=define
    useithreads=define
    usemultiplicity=define
    use64bitint=define
    use64bitall=define
    uselongdouble=undef
    usemymalloc=n
    default_inc_excludes_dot=define
    bincompat5005=undef
  Compiler:
    cc='/bin/gcc'
    ccflags ='-D_REENTRANT -g2 -O2 -m64 -march=native -fPIC -pthread -fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -I/usr/gnu/include -m64 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -DPERL_USE_SAFE_PUTENV'
    optimize='-O'
    cppflags='-I/export/home/jwalton/tmp/ok2delete/include -DNDEBUG -D_REENTRANT -g2 -O2 -m64 -march=native -fPIC -pthread -fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -I/usr/gnu/include'
    ccversion=''
    gccversion='4.8.2'
    gccosandvers=''
    intsize=4
    longsize=8
    ptrsize=8
    doublesize=8
    byteorder=12345678
    doublekind=3
    d_longlong=define
    longlongsize=8
    d_longdbl=define
    longdblsize=16
    longdblkind=3
    ivtype='long'
    ivsize=8
    nvtype='double'
    nvsize=8
    Off_t='off_t'
    lseeksize=8
    alignbytes=8
    prototype=define
  Linker and Libraries:
    ld='/bin/gcc'
    ldflags =' -L/export/home/jwalton/tmp/ok2delete/lib -m64 -Wl,-R,$ORIGIN/../lib -Wl,-R,/export/home/jwalton/tmp/ok2delete/lib -Wl,-z,now -Wl,-z,origin -fstack-protector -L/usr/local/lib -L/usr/gnu/lib -m64 '
    libpth=/export/home/jwalton/tmp/ok2delete/lib /usr/local/lib /usr/gcc/4.8/lib /usr/gcc/4.8/lib/gcc/i386-pc-solaris2.11/4.8.2/include-fixed /usr/lib /usr/gnu/lib /usr/ccs/lib
    libs=-lpthread -lsocket -lnsl -lgdbm -ldb -ldl -lm -lc
    perllibs=-lpthread -lsocket -lnsl -ldl -lm -lc
    libc=/lib/libc.so
    so=so
    useshrplib=true
    libperl=libperl.so
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs
    dlext=so
    d_dlsymun=undef
    ccdlflags='  -R /export/home/jwalton/tmp/ok2delete/lib/perl5/5.32.0/i86pc-solaris-thread-multi-64/CORE'
    cccdlflags='-fPIC'
    lddlflags=' -shared -m64 -L/export/home/jwalton/tmp/ok2delete/lib -Wl,-R,$ORIGIN/../lib -Wl,-R,/export/home/jwalton/tmp/ok2delete/lib -L/usr/local/lib -L/usr/gnu/lib -fstack-protector'

Characteristics of this binary (from libperl): 
  Compile-time options:
    HAS_TIMES
    MULTIPLICITY
    PERLIO_LAYERS
    PERL_COPY_ON_WRITE
    PERL_DONT_CREATE_GVSV
    PERL_IMPLICIT_CONTEXT
    PERL_MALLOC_WRAP
    PERL_OP_PARENT
    PERL_PRESERVE_IVUV
    PERL_USE_SAFE_PUTENV
    USE_64_BIT_ALL
    USE_64_BIT_INT
    USE_ITHREADS
    USE_LARGE_FILES
    USE_LOCALE
    USE_LOCALE_COLLATE
    USE_LOCALE_CTYPE
    USE_LOCALE_NUMERIC
    USE_LOCALE_TIME
    USE_PERLIO
    USE_PERL_ATOF
    USE_REENTRANT_API
  Built under solaris
  Compiled at Jan  9 2021 20:36:57
  %ENV:
    PERL5LIB="./lib"
  @INC:
    ./lib
    /export/home/jwalton/tmp/ok2delete/lib/perl5/site_perl/5.32.0/i86pc-solaris-thread-multi-64
    /export/home/jwalton/tmp/ok2delete/lib/perl5/site_perl/5.32.0
    /export/home/jwalton/tmp/ok2delete/lib/perl5/5.32.0/i86pc-solaris-thread-multi-64
    /export/home/jwalton/tmp/ok2delete/lib/perl5/5.32.0
noloader commented 3 years ago

Here is the error. It looks like something is not linking with -lgdbm.

ext/GDBM_File/t/fatal .......................................... #   Failed test 'use GDBM_File;'
#   at t/fatal.t line 24.
#     Tried to use 'GDBM_File'.
#     Error:  Can't load '../../lib/auto/GDBM_File/GDBM_File.so' for module GDBM_File: ld.so.1: perl: fatal: relocation error: file ../../lib/auto/GDBM_File/GDBM_File.so: symbol gdbm_errno: referenced symbol not found at ../../lib/XSLoader.pm line 93.
#  at ../../lib/GDBM_File.pm line 90.
# Compilation failed in require at t/fatal.t line 24.
# BEGIN failed--compilation aborted at t/fatal.t line 24.
Bareword "GDBM_WRCREAT" not allowed while "strict subs" in use at t/fatal.t line 39.
Execution of t/fatal.t aborted due to compilation errors.
# Looks like your test exited with 255 just after 1.
FAILED at test 1
ext/GDBM_File/t/gdbm ........................................... #   Failed test 'use GDBM_File;'
#   at ../../t/lib/dbmt_common.pl line 20.
#     Tried to use 'GDBM_File'.
#     Error:  Can't load '../../lib/auto/GDBM_File/GDBM_File.so' for module GDBM_File: ld.so.1: perl: fatal: relocation error: file ../../lib/auto/GDBM_File/GDBM_File.so: symbol gdbm_errno: referenced symbol not found at ../../lib/XSLoader.pm line 93.
#  at ../../lib/GDBM_File.pm line 90.
# Compilation failed in require at ../../t/lib/dbmt_common.pl line 20.
# BEGIN failed--compilation aborted at ../../t/lib/dbmt_common.pl line 20.
#   Failed test '(eval q{(GDBM_WRCREAT, GDBM_WRITER)})[0]'
#   at ../../t/lib/dbmt_common.pl line 24.
#          got: undef
#     expected: anything else
#   Failed test '(eval q{(GDBM_WRCREAT, GDBM_WRITER)})[1]'
#   at ../../t/lib/dbmt_common.pl line 25.
#          got: undef
#     expected: anything else
GDBM_File must define either a TIEHASH() or a new() method at ../../t/lib/dbmt_common.pl line 38.
Compilation failed in require at t/gdbm.t line 6.
# Tests were run but no plan was declared and done_testing() was not seen.
# Looks like your test exited with 255 just after 3.
FAILED at test 1

If someone can tell me the name of the makefile, I can probably use sed to add the link library.

perl-5.32.0$ find . -name '*GDBM*'
./lib/GDBM_File.pm
./lib/auto/GDBM_File
./lib/auto/GDBM_File/GDBM_File.so
./ext/GDBM_File
./ext/GDBM_File/GDBM_File.o
./ext/GDBM_File/GDBM_File.xs
./ext/GDBM_File/GDBM_File.pm
./ext/GDBM_File/GDBM_File.c
./ext/GDBM_File/GDBM_File.bs

And:

perl-5.32.0$ /usr/gnu/bin/nm lib/auto/GDBM_File/GDBM_File.so | grep gdbm
                 U gdbm_close
                 U gdbm_delete
                 U gdbm_errno
                 U gdbm_exists
                 U gdbm_fetch
                 U gdbm_firstkey
                 U gdbm_nextkey
                 U gdbm_open
                 U gdbm_reorganize
                 U gdbm_setopt
                 U gdbm_store
                 U gdbm_sync

And I added a libgdbm.pc so Perl would have compile and link information for the library. It did not help:

$ cat ~/tmp/ok2delete/lib/pkgconfig/libgdbm.pc 
prefix=/export/home/jwalton/tmp/ok2delete
exec_prefix=${prefix}
includedir=${prefix}/include
libdir=${prefix}/lib

Name: GNU dbm
Description: Database functions similar to UNIX dbm
URL: https://www.gnu.org.ua/software/gdbm/
Version: 1.19
Cflags: -I${includedir}
Libs: -L${libdir} -lgdbm

And:

$ find ~/tmp/ok2delete/lib -name 'lib*gdbm*'
/export/home/jwalton/tmp/ok2delete/lib/libgdbm.so.6.0.0
/export/home/jwalton/tmp/ok2delete/lib/libgdbm.a
/export/home/jwalton/tmp/ok2delete/lib/libgdbm.la
/export/home/jwalton/tmp/ok2delete/lib/pkgconfig/libgdbm.pc
/export/home/jwalton/tmp/ok2delete/lib/libgdbm.so.6
/export/home/jwalton/tmp/ok2delete/lib/libgdbm.so
noloader commented 3 years ago

Here is a similar report in Python: Issue 29440.

And I don't know if this is relevant from the GDBM manual: Compatibility with standard dbm and ndbm.

graygnuorg commented 3 years ago

gdbm_errno is a preprocessor macro defined in gdbm.h. If it appears in linker output, that means that the include file gbdm.h used during compilation does not match the libgdbm library used during linking. Most probably the gdbm.h file used belongs to an earlier version of gdbm (prior to version 1.14). If you have multiple versions of gdbm installed, make sure you use the right -I options in your CPPFLAGS.

jkeenan commented 3 years ago

Here is the error. It looks like something is not linking with -lgdbm.

ext/GDBM_File/t/fatal .......................................... #   Failed test 'use GDBM_File;'
#   at t/fatal.t line 24.
#     Tried to use 'GDBM_File'.
#     Error:  Can't load '../../lib/auto/GDBM_File/GDBM_File.so' for module GDBM_File: ld.so.1: perl: fatal: relocation error: file ../../lib/auto/GDBM_File/GDBM_File.so: symbol gdbm_errno: referenced symbol not found at ../../lib/XSLoader.pm line 93.
#  at ../../lib/GDBM_File.pm line 90.
# Compilation failed in require at t/fatal.t line 24.
# BEGIN failed--compilation aborted at t/fatal.t line 24.
Bareword "GDBM_WRCREAT" not allowed while "strict subs" in use at t/fatal.t line 39.
Execution of t/fatal.t aborted due to compilation errors.
# Looks like your test exited with 255 just after 1.
FAILED at test 1
ext/GDBM_File/t/gdbm ........................................... #   Failed test 'use GDBM_File;'
#   at ../../t/lib/dbmt_common.pl line 20.
#     Tried to use 'GDBM_File'.
#     Error:  Can't load '../../lib/auto/GDBM_File/GDBM_File.so' for module GDBM_File: ld.so.1: perl: fatal: relocation error: file ../../lib/auto/GDBM_File/GDBM_File.so: symbol gdbm_errno: referenced symbol not found at ../../lib/XSLoader.pm line 93.
#  at ../../lib/GDBM_File.pm line 90.
# Compilation failed in require at ../../t/lib/dbmt_common.pl line 20.
# BEGIN failed--compilation aborted at ../../t/lib/dbmt_common.pl line 20.
#   Failed test '(eval q{(GDBM_WRCREAT, GDBM_WRITER)})[0]'
#   at ../../t/lib/dbmt_common.pl line 24.
#          got: undef
#     expected: anything else
#   Failed test '(eval q{(GDBM_WRCREAT, GDBM_WRITER)})[1]'
#   at ../../t/lib/dbmt_common.pl line 25.
#          got: undef
#     expected: anything else
GDBM_File must define either a TIEHASH() or a new() method at ../../t/lib/dbmt_common.pl line 38.
Compilation failed in require at t/gdbm.t line 6.
# Tests were run but no plan was declared and done_testing() was not seen.
# Looks like your test exited with 255 just after 3.
FAILED at test 1

If someone can tell me the name of the makefile, I can probably use sed to add the link library.

perl-5.32.0$ find . -name '*GDBM*'
./lib/GDBM_File.pm
./lib/auto/GDBM_File
./lib/auto/GDBM_File/GDBM_File.so
./ext/GDBM_File
./ext/GDBM_File/GDBM_File.o
./ext/GDBM_File/GDBM_File.xs
./ext/GDBM_File/GDBM_File.pm
./ext/GDBM_File/GDBM_File.c
./ext/GDBM_File/GDBM_File.bs

And:

perl-5.32.0$ /usr/gnu/bin/nm lib/auto/GDBM_File/GDBM_File.so | grep gdbm
                 U gdbm_close
                 U gdbm_delete
                 U gdbm_errno
                 U gdbm_exists
                 U gdbm_fetch
                 U gdbm_firstkey
                 U gdbm_nextkey
                 U gdbm_open
                 U gdbm_reorganize
                 U gdbm_setopt
                 U gdbm_store
                 U gdbm_sync

And I added a libgdbm.pc so Perl would have compile and link information for the library. It did not help:

$ cat ~/tmp/ok2delete/lib/pkgconfig/libgdbm.pc 
prefix=/export/home/jwalton/tmp/ok2delete
exec_prefix=${prefix}
includedir=${prefix}/include
libdir=${prefix}/lib

Name: GNU dbm
Description: Database functions similar to UNIX dbm
URL: https://www.gnu.org.ua/software/gdbm/
Version: 1.19
Cflags: -I${includedir}
Libs: -L${libdir} -lgdbm

And:

$ find ~/tmp/ok2delete/lib -name 'lib*gdbm*'
/export/home/jwalton/tmp/ok2delete/lib/libgdbm.so.6.0.0
/export/home/jwalton/tmp/ok2delete/lib/libgdbm.a
/export/home/jwalton/tmp/ok2delete/lib/libgdbm.la
/export/home/jwalton/tmp/ok2delete/lib/pkgconfig/libgdbm.pc
/export/home/jwalton/tmp/ok2delete/lib/libgdbm.so.6
/export/home/jwalton/tmp/ok2delete/lib/libgdbm.so

Could this be the same problem reported in https://github.com/Perl/perl5/issues/16594?

graygnuorg commented 3 years ago

No, it is completely unrelated.

noloader commented 3 years ago

OK, so I added --enable-libgdbm-compat to my GDBM recipe. GDBM now provides two libraries - libgdbm.so and libgdbm_compat.so. Everything is installed in $HOME/tmp/ok2delete.

Building Perl now results in:

...
LD_LIBRARY_PATH=/export/home/jwalton/Build-Scripts/perl-5.32.0 ./miniperl -Ilib make_ext.pl lib/auto/NDBM_File/NDBM_File.so  MAKE="gmake" LIBPERL_A=libperl.so LINKTYPE=dynamic
Warning (mostly harmless): No library found for -lndbm
Warning (mostly harmless): No library found for -ldbm
Generating a Unix-style Makefile
Writing Makefile for NDBM_File
gmake[1]: Entering directory `/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/NDBM_File'
Running Mkbootstrap for NDBM_File ()
chmod 644 "NDBM_File.bs"
/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/NDBM_File/../../miniperl "-I../../lib" -MExtUtils::Command::MM -e 'cp_nonempty' -- NDBM_File.bs ../../lib/auto/NDBM_File/NDBM_File.bs 644
"../../miniperl" "-I../../lib" "../../lib/ExtUtils/xsubpp" -noprototypes -typemap '/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/NDBM_File/../../lib/ExtUtils/typemap' -typemap '/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/NDBM_File/typemap'  NDBM_File.xs > NDBM_File.xsc
mv NDBM_File.xsc NDBM_File.c
/bin/gcc -c   -D_REENTRANT -g2 -O2 -m64 -march=native -fPIC -pthread -fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -I/usr/gnu/include -m64 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -DPERL_USE_SAFE_PUTENV -Wall -Werror=pointer-arith -Wextra -Wc++-compat -Wwrite-strings -Werror=declaration-after-statement -O   -DVERSION=\"1.15\" -DXS_VERSION=\"1.15\" -fPIC "-I../.."   NDBM_File.c
rm -f ../../lib/auto/NDBM_File/NDBM_File.so
/bin/gcc  -shared -m64 -L/export/home/jwalton/tmp/ok2delete/lib -Wl,-R,RIGIN/../lib -Wl,-R,/export/home/jwalton/tmp/ok2delete/lib -L/usr/local/lib -L/usr/gnu/lib -fstack-protector  NDBM_File.o  -o ../../lib/auto/NDBM_File/NDBM_File.so  \
      \

chmod 755 ../../lib/auto/NDBM_File/NDBM_File.so
gmake[1]: Leaving directory `/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/NDBM_File'
LD_LIBRARY_PATH=/export/home/jwalton/Build-Scripts/perl-5.32.0 ./miniperl -Ilib make_ext.pl lib/auto/ODBM_File/ODBM_File.so  MAKE="gmake" LIBPERL_A=libperl.so LINKTYPE=dynamic
Warning (mostly harmless): No library found for -ldbm
Generating a Unix-style Makefile
Writing Makefile for ODBM_File
gmake[1]: Entering directory `/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/ODBM_File'
Running Mkbootstrap for ODBM_File ()
chmod 644 "ODBM_File.bs"
/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/ODBM_File/../../miniperl "-I../../lib" -MExtUtils::Command::MM -e 'cp_nonempty' -- ODBM_File.bs ../../lib/auto/ODBM_File/ODBM_File.bs 644
"../../miniperl" "-I../../lib" "../../lib/ExtUtils/xsubpp" -noprototypes -typemap '/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/ODBM_File/../../lib/ExtUtils/typemap' -typemap '/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/ODBM_File/typemap'  ODBM_File.xs > ODBM_File.xsc
mv ODBM_File.xsc ODBM_File.c
/bin/gcc -c   -D_REENTRANT -g2 -O2 -m64 -march=native -fPIC -pthread -fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -I/usr/gnu/include -m64 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -DPERL_USE_SAFE_PUTENV -Wall -Werror=pointer-arith -Wextra -Wc++-compat -Wwrite-strings -Werror=declaration-after-statement -O   -DVERSION=\"1.16\" -DXS_VERSION=\"1.16\" -fPIC "-I../.."   ODBM_File.c
ODBM_File.xs:12:19: fatal error: dbm.h: No such file or directory
 #  include <dbm.h>
                   ^
compilation terminated.
gmake[1]: *** [ODBM_File.o] Error 1
gmake[1]: Leaving directory `/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/ODBM_File'
gmake[1]: Entering directory `/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/ODBM_File'
/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/ODBM_File/../../miniperl "-I../../lib" -MExtUtils::Command::MM -e 'cp_nonempty' -- ODBM_File.bs ../../lib/auto/ODBM_File/ODBM_File.bs 644
/bin/gcc -c   -D_REENTRANT -g2 -O2 -m64 -march=native -fPIC -pthread -fwrapv -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -I/usr/gnu/include -m64 -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -DPERL_USE_SAFE_PUTENV -Wall -Werror=pointer-arith -Wextra -Wc++-compat -Wwrite-strings -Werror=declaration-after-statement -O   -DVERSION=\"1.16\" -DXS_VERSION=\"1.16\" -fPIC "-I../.."   ODBM_File.c
ODBM_File.xs:12:19: fatal error: dbm.h: No such file or directory
 #  include <dbm.h>
                   ^
compilation terminated.
gmake[1]: *** [ODBM_File.o] Error 1
gmake[1]: Leaving directory `/export/home/jwalton/Build-Scripts/perl-5.32.0/ext/ODBM_File'
Unsuccessful make(ext/ODBM_File): code=512 at make_ext.pl line 588.
gmake: *** [lib/auto/ODBM_File/ODBM_File.so] Error 2

It looks like some of my CFLAGS are being used, but my CPPFLAGS are not being used. Neither -I/export/home/jwalton/tmp/ok2delete/include -DNDEBUG are being used.

For completeness, here are the flags I am providing Perl's build system:

 AUTOCONF_BUILD: x86_64-pc-solaris2.11
PKG_CONFIG_PATH: /export/home/jwalton/tmp/ok2delete/lib/pkgconfig
       CPPFLAGS: -I/export/home/jwalton/tmp/ok2delete/include -DNDEBUG
        ASFLAGS: -Wa,--noexecstack
         CFLAGS: -g2 -O2 -m64 -march=native -fPIC -pthread
       CXXFLAGS: -g2 -O2 -m64 -march=native -fPIC -pthread
        LDFLAGS: -L/export/home/jwalton/tmp/ok2delete/lib -m64 -Wl,-R,'$ORIGIN/../lib' -Wl,-R,/export/home/jwalton/tmp/ok2delete/lib -Wl,-z,now -Wl,-z,origin
         LDLIBS: -ldl -lpthread

And to make matters worse, the build system is using random flags, like -I/usr/local/include -I/usr/gnu/include. I make no guarantees that the stuff in /usr/local are compatible with the system's warez. The only safe thing to use (unless I pass different flags) is -I/usr/include.

This is a big problem from the Perl build:

Generating a Unix-style Makefile

I essentially run Configure && make && make check && make install in a script. After Configure the script fixes various errors in makefiles. When Perl generates a makefile on the fly the script cannot fix it. Additionally, the script cannot fix errors in artifacts with non-standard names, like Makefile.SH.

I'm trying to understand what perl-5.32.0/make_ext.pl does. The file makes no sense to me because I don't read or write Perl. (The reason I am building Perl is, OpenSSL 1.1.x needs an updated Perl. The OpenSSL devs have no idea the trouble they cause when they introduces these spurious dependencies).

With that said, this is what a makefile recipe should look like. It will make most of the problems go away.

# User owns CPPFLAGS, CFLAGS,  CXXFLAGS, LDFLAGS, LDLIBS, etc. Never touch them.
# Perl owns the flags of the same name with a PERL_ prefix.
#   - PERL_CPPFLAGS comes from `-Acppflags`
#   - PERL_CFLAGS comes from `-Accflags`
#   - PERL_CXXFLAGS comes from `-Acxxflags`
#   - PERL_ASFLAGS comes from `-ascflags`
#   - PERL_LDFLAGS comes from `-Aldflags`
#   - PERL_LDLIBS comes from `-Aldlibs`
# Perl should put flags like -DPERL_CORE -D_REENTRANT in PERL_CPPFLAGS

# Compile
foo.o: foo.h
    $(CC) $(PERL_CPPFLAGS) $(CPPFLAGS) $(PERL_CFLAGS) $(CFLAGS) -c $<

# Link
foo: foo.o
    $(CC) $(PERL_CFLAGS) $(CFLAGS) $(PERL_LDFLAGS) $(LDFLAGS) foo.o -o $@ $(PERL_LDLIBS) $(LDLIBS)

# Link
foo.so: foo.o
    $(CC) -shared $(PERL_CFLAGS) $(CFLAGS) $(PERL_LDFLAGS) $(LDFLAGS) foo.o -o $@ $(PERL_LDLIBS) $(LDLIBS)

If I could get my hands on those makefiles I could fix them so they utilize flags properly. But I cannot at the moment because they are built on the fly.

And keep in mind it is my responsibility to provide the proper environment and flags to the build system. If I don't provide the proper environment or flags, then the build system should puke. And Perl should use what I provide.

And it may be worth mentioning... LDFLAGS like -Wl,-z,relro and -Wl,-z,now are security hardening flags. Ignoring them is probably CVE worthy.

graygnuorg commented 3 years ago

Jeremy, you don't need --enable-libgdbm-compat. As a matter of fact, chances are nobody does, nowadays. This option is for the unlikely case when someone needs to run antiquated software, designed to work with ndbm (or worse yet, with UNIX dbm) and to use gdbm as the underlying database.

graygnuorg commented 3 years ago

Let me reiterate, the reason for the reported failure is:

... symbol gdbm_errno: referenced symbol not found ...

as reported by linker. The fact that linker reports it means that the symbol gdbm_errno is present is the GDBM_File.so. Now, there is no such symbol in the libgdbm.so (nor in any other shared library provided by gdbm, libgdbm_compat.so included). During proper compilation, it is replaced by a call to gdbm_errno_location() function (assuming gdbm version >= 1.14 is in use). "Proper compilation" occurs when the compiler uses header file (gdbm.h) belonging to the same version as libgdm.so that would be used later at the linking phase. Thus, the reported behavior can occur only if the gdbm.h file used when compiling GDBM_File.so does not match the shared library libgdbm.so the dynamic linker picks up when loading the GDBM_File.so into the program.

You reported that you have libgdbm.so.6.0.0 in /export/home/jwalton/tmp/ok2delete/lib. The "current" number 6 means the library is from gdbm version 1.16 or later. Your host system, most probably, has an earlier version of gdbm installed. There are two possibilities: either the header file from the system-wide version was used during compilation and libgdbm.so.6.0.0 was used during loading phase (most likely), or vice-versa. In either case, it is a mismatch between header and the library. Check what header file is used during compilation and what library is used during loading.

noloader commented 3 years ago

@graygnuorg

you don't need --enable-libgdbm-compat. As a matter of fact, chances are nobody does, nowadays.

Thanks. I was working from the manual at https://www.gnu.org.ua/software/gdbm/manual/Compatibility.html. That might be good information to add to the manual.

noloader commented 3 years ago

@graygnuorg,

You reported that you have libgdbm.so.6.0.0 in /export/home/jwalton/tmp/ok2delete/lib. The "current" number 6 means the library is from gdbm version 1.16 or later. Your host system, most probably, has an earlier version of gdbm installed. There are two possibilities: either the header file from the system-wide version was used during compilation and libgdbm.so.6.0.0 was used during loading phase (most likely), or vice-versa. In either case, it is a mismatch between header and the library. Check what header file is used during compilation and what library is used during loading.

It looks like Perl does not honor CPPFLAGS, -Acppflags, ASFLAGS and -Aasflags. It looks like they are ignored.

tonycoz commented 3 years ago

Oops, closed by mistake

tonycoz commented 3 years ago

I essentially run Configure && make && make check && make install in a script. After Configure the script fixes various errors in makefiles. When Perl generates a makefile on the fly the script cannot fix it. Additionally, the script cannot fix errors in artifacts with non-standard names, like Makefile.SH.

To change the variables used in generating the Makefile you can update config.sh, then make Makefile (to regenerate the base makefile).

noloader commented 3 years ago

Thanks @tonycoz,

To change the variables used in generating the Makefile you can update config.sh, then make Makefile (to regenerate the base makefile).

So I am clear, can you provide the exact procedure? There are at least 4 config.sh If I am seeing things correctly.

noloader commented 3 years ago

@tonycoz,

To change the variables used in generating the Makefile you can update config.sh, then make Makefile (to regenerate the base makefile).

So this does not appear to work. Can you provide a concrete procedure?

Here's what I've tried... I loop over all the config.sh found by find. I cd into the relevant directory. I change config.sh. make Makefile fails:

Fixing config.sh
amigaos4/config.sh
# make Makefile executed from amigaos4/
make: *** No rule to make target `Makefile'.  Stop.

Here are the 4 config.sh I am finding:

$ find . -name config.sh
./Porting/config.sh
./amigaos4/config.sh
./symbian/config.sh
./config.sh
graygnuorg commented 3 years ago

Use the -D option to supply compiler and loader flags. E.g., assuming the libraries are located in ~/tmp/ok2delete/lib and includes in /tmp/ok2delete/include:

./Configure -des -Dusedevel -Dccflags=-I$HOME/tmp/ok2delete/include -Dldflags=-L$HOME/tmp/ok2delete/lib

Notice also, that you'll have to inform the dynamic loader about your non-standard library directory. Normally this is done by setting the LD_LIBRARY_PATH environment variable, e.g.:

export LD_LIBRARY_PATH=$HOME/tmp/ok2delete/lib

See man ld.so, for details.

tonycoz commented 3 years ago

To change the variables used in generating the Makefile you can update config.sh, then make Makefile (to regenerate the base makefile).

So I am clear, can you provide the exact procedure? There are at least 4 config.sh If I am seeing things correctly.

The only one that matters for you on Linux is the one generated by Configure in the build root.

symbian/config.sh and amigaos4/config.sh are for Symbian and Amiga respectively. Porting/config.sh is a sample.

noloader commented 3 years ago

Thanks @tonycoz. I think I may have this part done.

So I am clear, do I need to fix the ORIGIN-based paths in config_args and config_arg11? And for completeness, -Wl,-R,'$ORIGIN/../lib' is correct as a implicit variable. When they get written to a Makefile the $ needs to be escaped to $$. The caller is responsible for escaping them.

Thanks again.

$ grep -IR ORIGIN perl-5.32.0/config.sh perl-5.32.0/Makefile
perl-5.32.0/config.sh:config_args='-des -Dprefix=/home/jwalton/ok2delete -Dlibdir=/home/jwalton/ok2delete/lib -Dpkgconfig=/home/jwalton/ok2delete/lib/pkgconfig -Dcc=/usr/bin/cc -Dcxx=/usr/bin/g++ -Acppflags=-I/home/jwalton/ok2delete/include -DNDEBUG -Aasflags=-Wa,--noexecstack -Accflags=-I/home/jwalton/ok2delete/include -DNDEBUG -g2 -O2 -march=native -fPIC -pthread -Acxxflags=-I/home/jwalton/ok2delete/include -DNDEBUG -g2 -O2 -march=native -fPIC -pthread -Aldflags=-L/home/jwalton/ok2delete/lib -Wl,-R,'$ORIGIN/../lib' -Wl,-R,/home/jwalton/ok2delete/lib -Wl,--enable-new-dtags -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack -Wl,-z,origin -Aldlibs=-ldl -lpthread -Duseshrplib -Dusethreads -Dextras=FindBin Text Util ExtUtils Term Test HTTP'
perl-5.32.0/config.sh:config_arg11='-Aldflags=-L/home/jwalton/ok2delete/lib -Wl,-R,'$ORIGIN/../lib' -Wl,-R,/home/jwalton/ok2delete/lib -Wl,--enable-new-dtags -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack -Wl,-z,origin'
perl-5.32.0/config.sh:lddlflags='-shared -O2 -L/home/jwalton/ok2delete/lib -Wl,-R,$$ORIGIN/../lib -Wl,-R,/home/jwalton/ok2delete/lib -L/usr/local/lib -fstack-protector-strong'
perl-5.32.0/config.sh:ldflags=' -L/home/jwalton/ok2delete/lib -Wl,-R,$$ORIGIN/../lib -Wl,-R,/home/jwalton/ok2delete/lib -Wl,--enable-new-dtags -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack -Wl,-z,origin -fstack-protector-strong -L/usr/local/lib'
perl-5.32.0/Makefile:LDFLAGS =  -L/home/jwalton/ok2delete/lib -Wl,-R,$$ORIGIN/../lib -Wl,-R,/home/jwalton/ok2delete/lib -Wl,--enable-new-dtags -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack -Wl,-z,origin -fstack-protector-strong -L/usr/local/lib
perl-5.32.0/Makefile:CLDFLAGS =  -L/home/jwalton/ok2delete/lib -Wl,-R,$$ORIGIN/../lib -Wl,-R,/home/jwalton/ok2delete/lib -Wl,--enable-new-dtags -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack -Wl,-z,origin -fstack-protector-strong -L/usr/local/lib
perl-5.32.0/Makefile:LDDLFLAGS = -shared -O2 -L/home/jwalton/ok2delete/lib -Wl,-R,$$ORIGIN/../lib -Wl,-R,/home/jwalton/ok2delete/lib -L/usr/local/lib -fstack-protector-strong

And one other question... Why is Perl adding -L/usr/local/lib? That's a random directory that should not be present. I make no guarantees the stuff built/installed in /usr/local is compatible with what I am doing in $HOME/ok2delete. Is it possible to disable the "feature"?

tonycoz commented 3 years ago

And one other question... Why is Perl adding -L/usr/local/lib? That's a random directory that should not be present. I make no guarantees the stuff built/installed in /usr/local is compatible with what I am doing in $HOME/ok2delete. Is it possible to disable the "feature"?

You can supply a different set of paths to -Dloclibpth, for example:

$ ./Configure -des -Dloclibpth=/home/tony/local -Dusedevel
...
$ grep '^ldflags' config.sh
ldflags=' -fstack-protector-strong -L/home/tony/local'
ldflags_uselargefiles=''
jkeenan commented 3 years ago

Since we have not heard from the original poster since Jan 12 and since @tonycoz has provided useful suggestions, I propose that we close this ticket in 7 days unless we get an update.

jkeenan commented 3 years ago

Since we have not heard from the original poster since Jan 12 and since @tonycoz has provided useful suggestions, I propose that we close this ticket in 7 days unless we get an update.

Closing ticket as per schedule.