perl5.18.0 build-config seems to ignore some 64-bit types

[p5pRT]

Migrated from rt.perl.org#119537 (status was 'resolved')

Searchable as RT119537$

[p5pRT]

From perl-diddler@tlinx.org

Created by perl-diddler@tlinx.org

I know that some people manage to get this to build on SuSE\, but there seem to be some fundamental issues that I'm not sure how or why perl would be configured to use ...

I.e​:

Ishtar​:packages/build/perl-5.18> make |& tee ../../logs/perl518-a.log `sh cflags "optimize='-O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -Wall -pipe'" perlmini.o` -fPIC -DPERL_IS_MINIPERL -DPERL_EXTERNAL_GLOB perlmini.c   CCCMD = cc -DPERL_CORE -c -D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -Wall -pipe -Wall   In file included from perl.c​:33​:0​:   perl.h​:739​:14​: error​: conflicting types for ‘syscall’   EXTERN_C int syscall(int\, ...);   ^   In file included from perl.h​:726​:0\,   from perl.c​:33​:   /usr/include/unistd.h​:1080​:17​: note​: previous declaration of ‘syscall’ was here   extern long int syscall (long int __sysno\, ...) __THROW;

There were other compile warnings based on unused return values but the above error ends the build. It's *as if* (and this makes it look related\, maybe? to previous bug regarding mmap?) perl is using a 32-bit int to hold what should be a 64-bit quantity on a 64-bit machine.

The build doesn't get any farther than this when run single-job.

The config line I used coming into this was​:

Ishtar​:packages/build/perl-5.18> ./configure.gnu --prefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Dlibswanted="gdbm gdbm_compat" -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Duseshrplib=\'true\' $options |& tee perlconf.log

# for options =

-Doptimize='-g -ggdb -O2 -m64 -fasynchronous-unwind-tables -fbranch-target-load-optimize -fdelete-null-pointer-checks -fgcse-after-reload -fgcse-las -fgcse-sm -fgraphite-identity -fipa-pta -fivopts -floop-block -floop-flatten -floop-interchange -floop-strip-mine -flto -fmessage-length=0 -fpredictive-commoning -frename-registers -freorder-blocks-and-partition -ftree-loop-linear -ftree-loop-distribution -ftree-loop-distribute-patterns -ftree-loop-im -ftree-loop-ivcanon -ftree-vectorize -ftree-slp-vectorize -funswitch-loops -funwind-tables -fvariable-expansion-in-unroller -fvect-cost-model -fweb -march=native -Wno-unused-result -pipe' -Accflags='-DPERL_USE_SAFE_PUTENV' -Dotherlibdirs=/usr/lib/perl5/site_perl

Is it just coincidence that this another place (previous (see #119475)) where perl isn't using the libc compat definition of a type in a system call?

Perl Info

``` Flags: category=core severity=medium This perlbug was built using Perl 5.16.2 - Fri Feb 15 01:17:37 UTC 2013 It is being executed now by Perl 5.16.2 - Fri Feb 15 01:12:05 UTC 2013. Site configuration information for perl 5.16.2: Configured by abuild at Fri Feb 15 01:12:05 UTC 2013. Summary of my perl5 (revision 5 version 16 subversion 2) configuration: Platform: osname=linux, osvers=3.4.6-2.10-default, archname=x86_64-linux-thread-multi uname='linux build34 3.4.6-2.10-default #1 smp thu jul 26 09:36:26 utc 2012 (641c197) x86_64 x86_64 x86_64 gnulinux ' config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Dd_dbm_open -Duseshrplib=true -Doptimize=-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe -Accflags=-DPERL_USE_SAFE_PUTENV -Dotherlibdirs=/usr/lib/perl5/site_perl' hint=recommended, useposix=true, d_sigaction=define useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=define, use64bitall=define, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe', cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector' ccversion='', gccversion='4.7.2 20130108 [gcc-4_7-branch revision 195012]', gccosandvers='' intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16 ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='cc', ldflags =' -L/usr/local/lib64 -fstack-protector' libpth=/lib64 /usr/lib64 /usr/local/lib64 libs=-lm -ldl -lcrypt -lpthread perllibs=-lm -ldl -lcrypt -lpthread libc=/lib64/libc-2.17.so, so=so, useshrplib=true, libperl=libperl.so gnulibc_version='2.17' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E -Wl,-rpath,/usr/lib/perl5/5.16.2/x86_64-linux-thread-multi/CORE' cccdlflags='-fPIC', lddlflags='-shared -L/usr/local/lib64 -fstack-protector' Locally applied patches: @INC for perl 5.16.2: /home/law/bin/lib /usr/lib/perl5/site_perl/5.16.2/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.16.2 /usr/lib/perl5/vendor_perl/5.16.2/x86_64-linux-thread-multi /usr/lib/perl5/vendor_perl/5.16.2 /usr/lib/perl5/5.16.2/x86_64-linux-thread-multi /usr/lib/perl5/5.16.2 /usr/lib/perl5/site_perl/5.16.2/x86_64-linux-thread-multi /usr/lib/perl5/site_perl/5.16.2 /usr/lib/perl5/site_perl . Environment for perl 5.16.2: HOME=/home/law LANG=en_US.UTF-8 LANGUAGE (unset) LC_COLLATE=C LC_CTYPE=en_US.UTF-8 LD_LIBRARY_PATH=/usr/lib64/mpi/gcc/openmpi/lib64 LOGDIR (unset) PATH=/home/law/bin/lib:/sbin:/usr/local/sbin:/usr/lib64/mpi/gcc/openmpi/bin:/home/law/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/kde3/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin:.:/usr/lib/qt3/bin:/opt/dell/srvadmin/bin:/usr/sbin:/etc/local/func_lib:/home/law/lib PERL5OPT=-Mutf8 -CSA -I/home/law/bin/lib PERL_BADLANG (unset) SHELL=/bin/bash ```

[p5pRT]

From @tux

On Fri\, 30 Aug 2013 12​:41​:50 -0700\, Linda Walsh (via RT) \perlbug\-followup@​perl\.org wrote​:

I know that some people manage to get this to build on SuSE\, but there seem to be some fundamental issues that I'm not sure how or why perl would be configured to use ...

Which SUSE? I might want to try to reproduce. I have (devel) acces to these​:

Linux 2.6.31.14-0.6-desktop [openSUSE 11.2 (Emerald)] Linux 2.6.34.7-0.7-desktop [openSUSE 11.3 (Teal)] Linux 2.6.34.10-0.6-desktop [openSUSE 11.3 (Teal)] Linux 3.1.10-1.29-default [openSUSE 12.1 (Asparagus)] Linux 3.4.28-2.20-desktop [openSUSE 12.2 (Mantis)] Linux 3.4.47-2.38-desktop [openSUSE 12.2 (Mantis)] Linux 3.7.10-1.11-desktop [openSUSE 12.3 (Dartmouth)] Linux 3.7.10-1.16-desktop [openSUSE 12.3 (Dartmouth)]

13.1 will be released in November\, and it already has been tagged "Evergreen"

-- H.Merijn Brand http​://tux.nl Perl Monger http​://amsterdam.pm.org/ using perl5.00307 .. 5.19 porting perl5 on HP-UX\, AIX\, and openSUSE http​://mirrors.develooper.com/hpux/ http​://www.test-smoke.org/ http​://qa.perl.org http​://www.goldmark.org/jeff/stupid-disclaimers/

[p5pRT]

The RT System itself - Status changed from 'new' to 'open'

[p5pRT]

From perl-diddler@tlinx.org

On Fri Aug 30 12​:58​:23 2013\, hmbrand wrote​:

Linux 3.7.10-1.11-desktop [openSUSE 12.3 (Dartmouth)] Linux 3.7.10-1.16-desktop [openSUSE 12.3 (Dartmouth)]

---

I have factory packages from as late as June 18\, installed on top of my last working 12.3 where factory had newer packages.

kernel info​:

uname -a Linux Ishtar 3.9.8-Isht-Van #3 SMP PREEMPT Sun Jul 7 20​:43​:00 PDT 2013 x86_64 x86_64 x86_64 GNU/Linux

---

I usually try to keep up on a more recent vanilla kernel build than suse provides.

[p5pRT]

From @tux

On Fri\, 30 Aug 2013 13​:10​:21 -0700\, "Linda Walsh via RT" \perlbug\-followup@​perl\.org wrote​:

On Fri Aug 30 12​:58​:23 2013\, hmbrand wrote​:

Linux 3.7.10-1.11-desktop [openSUSE 12.3 (Dartmouth)] Linux 3.7.10-1.16-desktop [openSUSE 12.3 (Dartmouth)] ---- I have factory packages from as late as June 18\, installed on top of my last working 12.3 where factory had newer packages.

kernel info​:

uname -a Linux Ishtar 3.9.8-Isht-Van #3 SMP PREEMPT Sun Jul 7 20​:43​:00 PDT 2013 x86_64 x86_64 x86_64 GNU/Linux

--- I usually try to keep up on a more recent vanilla kernel build than suse provides.

Right\, so my Linux 3.7.10-1.11-desktop [openSUSE 12.3 (Dartmouth)] comes closest. Both are 64bit machines\, meaning 64bit builds are default

From scratch\, without using my Policy.sh …

$ wget http​://cpan.metacpan.org/authors/id/R/RJ/RJBS/perl-5.18.0.tar.bz2 $ tbz -sx perl-5.18.0.tar.bz2 $ cd perl-5.18.0 $ ./Configure -ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Dd_dbm_open -Duseshrplib=true -Doptimize="-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe" -Accflags=-DPERL_USE_SAFE_PUTENV -Dotherlibdirs=/usr/lib/perl5/site_perl : Now you must run 'make'.

If you compile perl5 on a different machine or from a different object directory\, copy the Policy.sh file from this object directory to the new one before you run Configure -- this will help you with most of the policy defaults. $ make -j6 : make[1]​: Leaving directory `/pro/3gl/CPAN/perl-5.18.0/x2p'

  Everything is up to date. Type 'make test' to run test suite. $ env TEST_JOBS=5 make test_harness : All tests successful. Files=2392\, Tests=685711\, 265 wallclock secs (62.51 usr 14.47 sys + 497.08 cusr 59.64 csys = 633.70 CPU) Result​: PASS

You had this​: --8\<--- `sh cflags "optimize='-O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -Wall -pipe'" perlmini.o` -fPIC -DPERL_IS_MINIPERL -DPERL_EXTERNAL_GLOB perlmini.c   CCCMD = cc -DPERL_CORE -c -D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -Wall -pipe -Wall -->8---

Where is that -m64 coming from? I see in the ticket that you set $options\, but it does not show in config_args from perl -V

And\, if you want a 64bit perl\, why not do it the way installation instructions tell you to​: -Duse64bitall (I personally also add -Duselongdouble)

-- H.Merijn Brand http​://tux.nl Perl Monger http​://amsterdam.pm.org/ using perl5.00307 .. 5.19 porting perl5 on HP-UX\, AIX\, and openSUSE http​://mirrors.develooper.com/hpux/ http​://www.test-smoke.org/ http​://qa.perl.org http​://www.goldmark.org/jeff/stupid-disclaimers/

[p5pRT]

From perl-diddler@tlinx.org

On Fri Aug 30 13​:43​:04 2013\, hmbrand wrote​:

Right\, so my Linux 3.7.10-1.11-desktop [openSUSE 12.3 (Dartmouth)] comes closest. Both are 64bit machines\, meaning 64bit builds are default

From scratch\, without using my Policy.sh …

$ wget http​://cpan.metacpan.org/authors/id/R/RJ/RJBS/perl- 5.18.0.tar.bz2 $ tbz -sx perl-5.18.0.tar.bz2 $ cd perl-5.18.0 $ ./Configure -ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Dd_dbm_open -Duseshrplib=true -Doptimize="-fmessage-length=0 -O2 -Wall -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -g -Wall -pipe" -Accflags=-DPERL_USE_SAFE_PUTENV -Dotherlibdirs=/usr/lib/perl5/site_perl : Now you must run 'make'.

---

I know if I don't bend over backwards to get the gdbm and gdbm_compat libs included\, it won't have a chance of compiling in dbm functionality.

If you compile perl5 on a different machine or from a different object directory\, copy the Policy.sh file from this object directory to the new one before you run Configure -- this will help you with most of the policy defaults. $ make -j6 : make[1]​: Leaving directory `/pro/3gl/CPAN/perl-5.18.0/x2p'

---

Unfortunately\, the perl make (like most "config"-based make processes) alters itself based on the configuration of your compile machine (most importantly\, what packages and features you have installed).

  Theoretically\, if you don't have the pertinent "devel" packages loaded you shouldn't even be getting as far as you are. So I'm assuming you have some set of 'devel' rpms loaded with some headers&libs -- but the presence/absence of many different configurations is dependent on "which" set of 'devel' packages (headers&libs) you have installed.

Starting in about SuSE 11.4 or 12.0\, suse switched over to using disposable build environments that only include the necessary packages needed to build the specific package being built. And by necessary packages ('needs'\, meaning for 1 running configuration with unspecified functionality). However\, some things\, like a running kernel are assumed\, but kernel versions\, for example\, usually are not.

Same goes for many other "assumed to be present" utils\, though the list of assumed packages has been shrinking as more dependencies are added to the .spec file.

You had this​: --8\<--- `sh cflags "optimize='-O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -Wall -pipe'" perlmini.o` -fPIC -DPERL_IS_MINIPERL -DPERL_EXTERNAL_GLOB perlmini.c CCCMD = cc -DPERL_CORE -c -D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -Wall -pipe -Wall -->8---

Where is that -m64 coming from? I see in the ticket that you set $options\, but it does not show in config_args from perl -V

---

The config_args from perl-V is from the perl installed on my system that is being used to send the report -- which has next to nil to do with the perl I'm generating... Funny\, as I was submitting this I thought about the incongruity as I submitted this bug -- thinking what does this config information have to do with the config I'm building with?

I don't think the m64 is necessary and don't remember why it was there -- a historical carryover from some early 64-bit release when there was uncertainty about whether the 32 or 64 bit model was begin used as a target.

It was one of the ones I copied from the existing line that was in use in my /etc/rmprc -- to which I add optimization options above and beyond the default of O2\, but \< O3 to try to add in all options that may take more compile time\, but generally don't add codesize.

... ah \, the m64 option comes\, originally\, from the default rpmrc configuration in /usr/lib/rpm/rpmrc Where the default x86_64 optflags = optflags​: x86_64 -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables

I have added to that over the years\, and my opt flags looks like​:

optflags​: -g -ggdb -O2 -m64 -fasynchronous-unwind-tables \   -fbranch-target-load-optimize -fdelete-null-pointer-checks \   -fgcse-after-reload -fgcse-las -fgcse-sm -fgraphite-identity \   -fipa-pta -fivopts -floop-block -floop-flatten \   -floop-interchange -floop-strip-mine -flto -fmessage-length=0 \   -fpredictive-commoning -frename-registers \   -freorder-blocks-and-partition -ftree-loop-linear \   -ftree-loop-distribution -ftree-loop-distribute-patterns \   -ftree-loop-im -ftree-loop-ivcanon -ftree-vectorize \   -ftree-slp-vectorize -funswitch-loops -funwind-tables \   -fvariable-expansion-in-unroller -fvect-cost-model \   -fweb -march=native

And\, if you want a 64bit perl\, why not do it the way installation instructions tell you to​: -Duse64bitall (I personally also add -Duselongdouble)

---

I would too! I'm trying to use a build based on the suse .spec file\, since I know that not using the spec file will fail\, in at least the DB area.

So I am trying to do as "SuSE suggests -- and trying to build based on the .spec file"....

That said\, when I build only using the options and invocation method as listed by you in this bug report\,   *it builds* (gets farther than with suse spec file) but does not past the make test phase.

The tests fail in the dbm related tests *AND* fail in some CPAN related tests in the same way that "invalid closed"[sic] bug 119493 described in trying to access cpan.

I am attaching the suse perl spec file I was using (done an rpmbuild -bp) to unpack it into a dir and was doing the build by-hand to more closely watch each step. Also enclosing the test output.

Can provide the output from configure as well if that is of usefulness...

Am going to look at why there is a difference in the 'make' for the suse perl spec file and options I used and see if that uncovers anything.

[p5pRT]

From perl-diddler@tlinx.org

perl.spec

[p5pRT]

From perl-diddler@tlinx.org

p18test.log

[p5pRT]

From @doughera88

On Fri\, Aug 30\, 2013 at 12​:41​:50PM -0700\, Linda Walsh wrote​:

# New Ticket Created by Linda Walsh # Please include the string​: [perl #119537] # in the subject line of all future correspondence about this issue. # \<URL​: https://rt-archive.perl.org/perl5/Ticket/Display.html?id=119537 >

This is a bug report for perl from perl-diddler@​tlinx.org\, generated with the help of perlbug 1.39 running under perl 5.16.2.

----------------------------------------------------------------- [Please describe your issue here]

I know that some people manage to get this to build on SuSE\, but there seem to be some fundamental issues that I'm not sure how or why perl would be configured to use ...

I.e​:

Ishtar​:packages/build/perl-5.18> make |& tee ../../logs/perl518-a.log `sh cflags "optimize='-O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -Wall -pipe'" perlmini.o` -fPIC -DPERL_IS_MINIPERL -DPERL_EXTERNAL_GLOB perlmini.c CCCMD = cc -DPERL_CORE -c -D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -DPERL_USE_SAFE_PUTENV -O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -Wall -pipe -Wall In file included from perl.c​:33​:0​: perl.h​:739​:14​: error​: conflicting types for ‘syscall’ EXTERN_C int syscall(int\, ...); ^ In file included from perl.h​:726​:0\, from perl.c​:33​: /usr/include/unistd.h​:1080​:17​: note​: previous declaration of ‘syscall’ was here extern long int syscall (long int __sysno\, ...) __THROW;

This particular problem likely arose because Configure couldn't figure out that your system had a valid syscall prototype. I don't know why Configure had that problem\, and I have been unable to reproduce it.

The fact that -DPERL_USE_SAFE_PUTENV appears 10 times suggests that this might not have been a clean build from a fresh source tree. The fact that a variety of '-floop-*' options appear in your configure command below\, but not in the command above suggest to me that the command above was not generated from the configure command below.

In short\, I can't reproduce this and really don't understand how you got there either. It would not surprise me if there were a problem with some of the Configure prototype tests under some unusual conditions\, but I can't guess precisely what those conditions might be\, and you haven't given us enough information for us to determine that either.

I recommend the following​:

1. Start with a clean source tree.

2. Figure out the *minimal* Configure command line necessary to reproduce the problem. Remove config.sh and Policy.sh between each trial.

3. Post the *minimal* Configure command line\, the *output* of the ./myconfig script\, and the output of 'grep syscall config.sh'. That might help us reproduce or diagnose the problem.

--   Andy Dougherty doughera@​lafayette.edu

[p5pRT]

From perl-diddler@tlinx.org

On Fri Aug 30 18​:45​:24 2013\, doughera wrote​:

This particular problem likely arose because Configure couldn't figure out that your system had a valid syscall prototype. I don't know why Configure had that problem\, and I have been unable to reproduce it.

The fact that -DPERL_USE_SAFE_PUTENV appears 10 times suggests that this might not have been a clean build from a fresh source tree.

---

That might have had to do with some excess merging of ENV var that contained that with the ENV-options string\, though never have seen that many repetitions.

The fact that a variety of '-floop-*' options appear in your configure command below\, but not in the command above suggest to me that the command above was not generated from the configure command below.

In short\, I can't reproduce this and really don't understand how you got there either. It would not surprise me if there were a problem with some of the Configure prototype tests under some unusual conditions\, but I can't guess precisely what those conditions might be\, and you haven't given us enough information for us to determine that either.

I recommend the following​:

1. Start with a clean source tree.

---

Everything after the 1st build was from a fresh tree.

I.e when Merijin showed their sequence of untaring and building\, I just went to my work directory and untar'ed the tar image there. Vs. trying to get it to work in the "package" directory where I'm using a SuSE rpm .spec file that they would be using\, but likely in a freshly formatted build-container.

2. Figure out the *minimal* Configure command line necessary to reproduce the problem. Remove config.sh and Policy.sh between each trial.

---

  I wasn't sure of what the minimal files to remove were\, and untarring the files take about 2 seconds\, so was easier to go that route. Have tried about 5 different ways and seem to get further in my tools dir (building from tar) vs. package(bld from spec).

I also tried to build from the spec file by removing various patches that are included\, but I think I'm doing as much harm as good in doing that (temporary harm\, in that I'm commenting out patch lines that I can comment back in).

Of the ones built from tar\, as I mentioned in response to merijin\, I was able to get those to build but not test -- with the test results included in this report. The failure of the DBtests has been a persistent failure in perl tests for at least a year or more at this point (suse's answer build​: from rpm on our build machines; dont' expect to use an installed suse machine for making builds...we don't support it).

The CPAN test failures seen in the test output are relatively new and first appeared in "invalid" bug 119493 where I mentioned it with other problems in 5.18.

3. Post the *minimal* Configure command line\, the *output* of the ./myconfig script\, and the output of 'grep syscall config.sh'. That might help us reproduce or diagnose the problem.

---

My last built with the rpm fails here​:

/var/tmp/rpm-tmp.RUgHm8#50> read f /var/tmp/rpm-tmp.RUgHm8#51> /usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/bin/perl -I/usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/lib/perl5/5.18.0 -I/usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/lib/perl5/5.18.0/x86_64-linux-thread-multi /usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/bin/h2ph -d /usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/lib/perl5/vendor_perl/5.18.0/x86_64-linux-thread-multi linux/map_to_7segment.h linux/map_to_7segment.h -> linux/map_to_7segment.ph /var/tmp/rpm-tmp.RUgHm8#50> read f /var/tmp/rpm-tmp.RUgHm8#53> popd /usr/src/packages/BUILD/perl-5.18

/var/tmp/rpm-tmp.RUgHm8#54> gcc -print-file-name=include /var/tmp/rpm-tmp.RUgHm8#54> d=/usr/lib64/gcc/x86_64-suse-linux/4.8/include /var/tmp/rpm-tmp.RUgHm8#55> test -f /usr/lib64/gcc/x86_64-suse-linux/4.8/include/stdarg.h /var/tmp/rpm-tmp.RUgHm8#55> cd /usr/lib64/gcc/x86_64-suse-linux/4.8/include /var/tmp/rpm-tmp.RUgHm8#55> /usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/bin/perl -I/usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/lib/perl5/5.18.0 -I/usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/lib/perl5/5.18.0/x86_64-linux-thread-multi /usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/bin/h2ph -d /usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/lib/perl5/vendor_perl/5.18.0/x86_64-linux-thread-multi stdarg.h stddef.h float.h stdarg.h -> stdarg.ph stddef.h -> stddef.ph float.h -> float.ph /var/tmp/rpm-tmp.RUgHm8#57> rm /usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/lib/perl5/5.18.0/Pod/Perldoc/ToTk.pm /var/tmp/rpm-tmp.RUgHm8#63> /usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/bin/perl -e '$r=chr(128)."\\x{100}";/$r/' /var/tmp/rpm-tmp.RUgHm8#65> /usr/src/packages/BUILDROOT/perl-5.18-0.0.x86_64/usr/bin/perl -e '/\6666666666/' /var/tmp/rpm-tmp.RUgHm8​: line 65​: 17540 Segmentation fault (core dumped) $RPM_BUILD_ROOT/usr/bin/perl -e '/\6666666666/' error​: Bad exit status from /var/tmp/rpm-tmp.RUgHm8 (%install)

RPM build errors​:   Bad exit status from /var/tmp/rpm-tmp.RUgHm8 (%install)

---

more myconfig #!/bin/sh

# This script is designed to provide a handy summary of the configuration # information being used to build perl. This is especially useful if you # are requesting help from comp.lang.perl.misc on usenet or via mail.

# Note that the text lines /^Summary of/ .. /^\s*$/ are copied into Config.pm. cat \<\<'!NO!SUBS!' Summary of my perl5 (revision 5 version 18 subversion 0) configuration​:  
  Platform​:   osname=linux\, osvers=3.9.11-isht-van\, archname=x86_64-linux-thread-multi   uname='linux ishtar 3.9.11-isht-van #2 smp preempt sat aug 31 15​:25​:04 pdt 2013 x86_64 x86_64 x86_64 gnulinux '   config_args='-ds -e -Dprefix=/usr -Dvendorprefix=/usr -Dinstallusrbinperl -Dusethreads -Di_db -Di_dbm -Di_ndbm -Di_gdbm -Dd_dbm_open -Doptimize=-O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -pipe -Accflags=-DPERL_USE_SAFE_PUTENV -Dotherlibdirs=/usr/lib/perl5/site_perl'   hint=recommended\, useposix=true\, d_sigaction=define   useithreads=define\, usemultiplicity=define   useperlio=define\, d_sfio=undef\, uselargefiles=define\, usesocks=undef   use64bitint=define\, use64bitall=define\, uselongdouble=undef   usemymalloc=n\, bincompat5005=undef   Compiler​:   cc='cc'\, ccflags ='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'\,   optimize='-O2 -g -m64 -fmessage-length=0 -D_FORTIFY_SOURCE=2 -fstack-protector -funwind-tables -fasynchronous-unwind-tables -pipe'\,   cppflags='-D_REENTRANT -D_GNU_SOURCE -DPERL_USE_SAFE_PUTENV -fno-strict-aliasing -pipe -fstack-protector'   ccversion=''\, gccversion='4.8.0'\, gccosandvers=''   intsize=4\, longsize=8\, ptrsize=8\, doublesize=8\, byteorder=12345678   d_longlong=define\, longlongsize=8\, d_longdbl=define\, longdblsize=16   ivtype='long'\, ivsize=8\, nvtype='double'\, nvsize=8\, Off_t='off_t'\, lseeksize=8   alignbytes=8\, prototype=define   Linker and Libraries​:   ld='cc'\, ldflags =' -L/usr/local/lib64 -fstack-protector'   libpth=/lib64 /usr/lib64 /usr/local/lib64   libs=-lm -ldl -lcrypt -lpthread   perllibs=-lm -ldl -lcrypt -lpthread   libc=/lib64/libc-2.17.so\, so=so\, useshrplib=false\, libperl=libperl.a   gnulibc_version='2.17'   Dynamic Linking​:   dlsrc=dl_dlopen.xs\, dlext=so\, d_dlsymun=undef\, ccdlflags='-Wl\,-E'   cccdlflags='-fPIC'\, lddlflags='-shared -L/usr/local/lib64 -fstack-protector'

!NO!SUBS!

---

I don't know if it is worth adding in the grep for syscall stuff at this point\, as this is failing in a different way -- perhaps something was getting re-used in the old build dir that wasn't compat with current headers -- but the above build and a few others w/rpm and about 5-6 from the tar\, were all done in clean directories (freshly untar'ed)...

I can forget about the ".rpm" not building -- I used to build from tar images -- only after problems starting with perl5.16 under suse 12.0 and later\, that I first tried to address through the suse email lists\, did I start trying to make it work with rpm-- as the suse folks recommended starting with their rpm's (double edged sword).

The build w/o the rpm -- fresh untar gets the test results already attached (fails in db & CPAN)...

FWIW\, I find it hard to get a reliable\, reproducible\, controlled build\, as I'm used to specifying all of the configure options in a shell script and calling it from there\, but most of the options aren't documented as to how to call them from a shell script.

i.e. for squid\, for example (with majority lines elided).

#!/bin/bash pkg=squid . /home/tools/config.include ... CFLAGS="-g -m64 -fPIC -fmessage-length=0 -O2 -march=native" ...   CFLAGS="$CFLAGS" CCFLAGS="$CCFLAGS" \   LDFLAGS="$LDFLAGS" configure \ ...   --with-logdir=/var/log/$pkg \ ...   --enable-kill-parent-hack \   --enable-linux-netfilter \   --enable-ltdl-install \ ...   --enable-ssl

---

With perl\, I wasn't sure where\, but I noticed it kept state info\, so I wrote a script to rebuild it from the tar image\, fresh each time -- that automatically unpacks the latest tar image in my tools/perl dir...

---

I.e. usually I start with a clean perl-dir each build\, because it doesn't run the same way each time unless I start afresh (due to the various state files it keeps around).

----Just so you know\, I'm not kidding --- Ishtar​:tools/perl> more remake_perl_from_tar #!/bin/bash shopt -s extglob if (($#\<1)); then   declare -a s=( $(echo perl-5.??.[0-9].t*@​(gz|bz2|xz|ar) ) )   declare -i numsrc=${#s[*]}   if ((numsrc>1)); then   echo "Found more than one perl source; must specify which to use (5.16*\, etc)"   exit 1   elif ((numsrc==0)); then   echo "Found no source to build (perl-5.\.x.t*\)"   exit 2   fi   src="${s[0]}" else   src="$1";q=1   if [[ ! -f $src ]]; then   tf="$(echo perl-$src.t* )"   if [[ -f $tf ]]; then   q=0   src=$tf   else #try liberal   tf="$(echo perl*$src*)"   src="$tf"; q=1 #question   fi   fi fi if [[ ! -f $src || ! -r $src ]]; then   echo "Can't read source from $src"   exit 3 fi if [[ -f $src ]]; then   type="$(file -z --mime-type "$src"|cut -d\ -f2)"   type=${type#application/}   if [[ $type != x-tar ]]; then   echo "File $src doesn't seem to be right type."   echo "Expecting x-tar\, found \"$type\""   exit 4   fi fi if ((q)); then   echo "Had to be a little liberal w/your version#..."   echo "Using $src as the source file? Is that ok? (y/n)"   ans=""   while [[ $ans != "y" ]]; do   read -p "Using $src as the source file? Is that ok? (y/n) " ans   if [[ $ans == "n" ]]; then exit 5; fi   done fi dir=${src%%.t*} if [[ -d "$dir" ]]; then   echo "rm -fr $dir"   sleep .5   rm -fr "$dir" fi tar xaf "$src" || {   echo "tar exited with status $?; stopping" }

dir=${src%%.t*} if [[ ! -d "$dir" ]]; then   echo "don't know what directory to start in"   echo "I thought it was \"$dir\"" fi

log=/tmp/makeperl-$$.log

{   cd "$dir" && {   sh Configure -de &&   make &&   make test   status=$?   }

  if (($status==0)) ; then echo "perl make & tested ok"; exit 0; fi

  echo "Bad status \"$status\" on exit"   echo "Make NOT OKAY!" } >& "$log" &

echo "Make is running. To monitor the log do​:" echo "tail -f \"$log\""

---

So how should I move forward -- the DB problem has been a thorn for some time. The CPAN test fails in the test phase are new w/5.18.

Thanks! ;-)

 

[p5pRT]

From @tux

On Sat\, 31 Aug 2013 21​:50​:44 -0700\, "Linda Walsh via RT" \perlbug\-followup@&#8203;perl\.org wrote​:

Of the ones built from tar\, as I mentioned in response to merijin\, I was able to get those to build but not test -- with the test results included in this report. The failure of the DBtests has been a persistent failure in perl tests for at least a year or more at this point (suse's answer build​: from rpm on our build machines; dont' expect to use an installed suse machine for making builds...we don't support it).

Shall I call you Lindia?

Does this help?

--8\<--- INSTALL =item error​: too few arguments to function 'dbmclose'

Building ODBM_File on some (Open)SUSE distributions might run into this error\, as the header file is broken. There are two ways to deal with this

1. Disable the use of ODBM_FILE

  Configure ... -Dnoextensions=ODBM_File

2. Fix the header file\, somewhat like this​:

  --- a/usr/include/dbm.h 2010-03-24 08​:54​:59.000000000 +0100   +++ b/usr/include/dbm.h 2010-03-24 08​:55​:15.000000000 +0100   @​@​ -59\,4 +59\,4 @​@​ extern datum firstkey __P((void));

  extern datum nextkey __P((datum key));

  -extern int dbmclose __P((DBM *));   +extern int dbmclose __P((void)); -->8---

FWIW my 12.3 has

$ grep dbmclose /usr/include/dbm.h extern int dbmclose (void); $ rpm -qf /usr/include/dbm.h gdbm-devel-1.10-2.2.1.i586

I can forget about the ".rpm" not building -- I used to build from tar images -- only after problems starting with perl5.16 under suse 12.0 and later\, that I first tried to address through the suse email lists\, did I start trying to make it work with rpm-- as the suse folks recommended starting with their rpm's (double edged sword).

The build w/o the rpm -- fresh untar gets the test results already attached (fails in db & CPAN)...

FWIW\, I find it hard to get a reliable\, reproducible\, controlled build\, as I'm used to specifying all of the configure options in a shell script and calling it from there\, but most of the options aren't documented as to how to call them from a shell script.

I read the mail trice and still did not see if you tried the "minimal" Configure as Andy and I suggested.

unpack from tar\, then e.g.

$ ./Configure -Duse64bitall -des

The general top-level script you are looking for to make "reliable" builds in *your* environment is called Policy.sh\, and it is described in INSTALL

--8\<--- INSTALL =head2 Site-wide Policy settings

After Configure runs\, it stores a number of common site-wide "policy" answers (such as installation directories) in the Policy.sh file. If you want to build perl on another system using the same policy defaults\, simply copy the Policy.sh file to the new system's perl build directory\, and Configure will use it. This will work even if Policy.sh was generated for another version of Perl\, or on a system with a different architecture and/or operating system. However\, in such cases\, you should review the contents of the file before using it​: for example\, your new target may not keep its man pages in the same place as the system on which the file was generated.

Alternatively\, if you wish to change some or all of those policy answers\, you should

  rm -f Policy.sh

to ensure that Configure doesn't re-use them.

Further information is in the Policy_sh.SH file itself.

If the generated Policy.sh file is unsuitable\, you may freely edit it to contain any valid shell commands. It will be run just after the platform-specific hints files. -->8---

As an example\, I'll share my stripped down Policy.sh which I use across OS's. I use it like this​:

untar source $ cd perl-5.18.0 $ cp ../Policy.sh . $ ./Configure ...

My install location is /pro

--8\<--- Policy.sh #!/usr/bin/sh

# put this file in the perl source tree and run 'Configure -ds'

man1dir=/pro/local/man/man1 man3dir=/pro/local/man/man3

case "$versiononly" in   '') versiononly=$undef   if [ -x /pro/bin/perl ]; then   if [ `/pro/bin/perl -e '$v="5";while(\<>){m/#define\s+PERL_(?​:SUB)?VERSION\s+(\d+)/and$v.=".$1"}$v=~s/\.(.)\./.00\1/;print STDERR "$v \<=> $]\n";$]\<$v?1​:0' $src/patchlevel.h` ]; then   versiononly=$define   installusrbinperl='undef'   fi   fi   ;;   esac prefix=/pro case "$cc" in   '') cc=cc ;;   esac cc="ccache $cc" perladmin='my-email-address' cf_email='my-email-address'

[ "X$OSTYPE" = "X" ] && OSTYPE="$osname" echo "===== PROCURA Policy for $OSTYPE/$cc ========================">&2

# The -DDEBUGGING part will be stripped out optionally later # It is easier to strip than to add ccflags='-fPIC -DDEBUGGING'

extras='none' inc_version_list='none' noextensions=ODBM_File

libsdirs=' /lib /pro/local/lib' ldflags='-L/pro/local/lib'

locincpth='/pro/local/include' loclibpth='/pro/local/lib'

awk='gawk' csh='tcsh'

case "$OSTYPE" in   aix)   case "$cc" in   *gcc*) if [ `uname -r` = 2 ]; then   : AIX 4.2 does not support these options   else   if [ "X$use64bitall" = "X" ]; then   ccflags="-maix32 $ccflags"   else   ccflags="-maix64 $ccflags"   fi   fi   ;;   *) if [ "$useithreads" = "define" ]; then   cc=xlc_r   else   cc=xlc   fi   #optimize='-O2'   ;;   esac   ;;

  hpux)   case "$cc" in   *gcc*) if [ "X$use64bitint" = "X" -a "X$use64bitall" = "X" ]; then   true   else   cc=gcc64   ldflags=''   fi   case `uname -r` in   B.10.20) ccflags="-mpa-risc-1-1 $ccflags" ;;   *) ccflags="-mpa-risc-2-0 $ccflags" ;;   esac   echo " Using" `which $cc` >&4   echo 'int main(){long l;printf("%d\\n"\,sizeof(l));}'>try.c   $cc -o try $ccflags $ldflags try.c   if [ "`try`" = "8" ]; then   echo "Your C compiler ($cc) is 64 bit native!" >&4   PATH=/usr/local/pa20_64/bin​:$PATH   locincpth=' /usr/local/pa20_64/include'   libsdirs=' /usr/local/pa20_64/lib /usr/lib/pa20_64'   libdirs=' /usr/local/pa20_64/lib /usr/lib/pa20_64'   loclibpth='/usr/local/pa20_64/lib'   ldflags=''   fi   rm -f try try.c   ;;   *) case `uname -r` in   B.10.20) ccflags="$ccflags +DAportable" ;;   esac   # -fast = +O3 +Onolooptransform +Olibcalls +FPD +Oentrysched +Ofastaccess'   # optimize='+O2 +Onolooptransform +Olibcalls +FPD +Onolimit'

  # For Oracle\, will fail without perlio in threads   # 5.8.0 and up have useperlio in default   ;;   esac   # For Oracle   libswanted="cl pthread $libswanted"   ;;

  osf1)   ccflags="$ccflags -std -D_INTRINSICS -D_INLINE_INTRINSICS"   optimize='-O2'   useshrplib='false'   ;;

  linux)   awk='awk'   _awk='/usr/bin/awk'   full_awk='/usr/bin/awk'   csh='tcsh'   _csh='/usr/bin/tcsh'   full_csh='/usr/bin/tcsh'   ;;   esac -->8---

Add as many OS's as you support this way (I stripped a few).

A Policy.sh should be written to be ably to create reproducible builds. ENV vars and options often do not mix very well with existing leftovers from previous builds.

So\, the way to go for you I guess is to create a Policy.sh which holds all the things you do want to deviate from the default Configure settings\, getting to a point where it will always fail. Then post that Policy.sh here and get us to see where things go wrong

-- H.Merijn Brand http​://tux.nl Perl Monger http​://amsterdam.pm.org/ using perl5.00307 .. 5.19 porting perl5 on HP-UX\, AIX\, and openSUSE http​://mirrors.develooper.com/hpux/ http​://www.test-smoke.org/ http​://qa.perl.org http​://www.goldmark.org/jeff/stupid-disclaimers/

[p5pRT]

From @doughera88

On Sat\, Aug 31\, 2013 at 09​:50​:44PM -0700\, "Linda Walsh via RT" wrote​:

On Fri Aug 30 18​:45​:24 2013\, doughera wrote​:

I recommend the following​:

1. Start with a clean source tree. 2. Figure out the *minimal* Configure command line necessary to reproduce the problem.

Have tried about 5 different ways and seem to get further in my tools dir (building from tar) vs. package(bld from spec).

I don't need to hear about 5 ways. I need to see a single way\, from a clean perl-5.18.0 distribution\, that I can try to reproduce. I don't want to see the failure of an rpm build. I don't use rpm.

[lots of irrelevant lines deleted.]

So how should I move forward -- the DB problem has been a thorn for some time. The CPAN test fails in the test phase are new w/5.18.

You move forward exactly as I said in my previous email.

First\, you provide us with the *minimal* Configure command line necessary to reproduce the problem in a clean perl directory.

Second\, you provide the output of the ./myconfig script from that directory.

Third\, you provide the exact error messages or test failures that result.

Only then can someone else hope to understand your problem\, try to reproduce it\, and possibly even suggest ways to fix it.

--   Andy Dougherty doughera@​lafayette.edu

[p5pRT]

From perl-diddler@tlinx.org

On Sun Sep 01 15​:16​:33 2013\, doughera wrote​:

First\, you provide us with the *minimal* Configure command line necessary to reproduce the problem in a clean perl directory.

---

Going from tar and Configure\, the problem I see now has to do with the DB feature; the dir where the source is built is recreated at the start of my make.

I had been using Configure -da\, but Merijn suggested​: Configure -Duse64bitall -des\, so altered my script to call that.

In answer to Merijn\, RE\, the suse-specific info provided​: I checked the ".h" file indicated and it was already prototyped as having no params(void)​:

grep dbmclose /usr/include/dbm.h extern int dbmclose (void);

which\, unfortunately indicates that the suse-specific problem mentioned\, is not likely a contributing cause to anything I'm seeing.

Second\, you provide the output of the ./myconfig script from that directory.

---

myconfig script attached.

Third\, you provide the exact error messages or test failures that result.

---

Among others​:

lib/AnyDBM_File ............................................... #
Failed test 'Tie' # at ../lib/AnyDBM_File.t line 25. FAILED at test 1 .... Failed 10 tests out of 2254\, 99.56% okay.   ../cpan/Memoize/t/errors.t   ../cpan/Memoize/t/tie_gdbm.t   ../cpan/Memoize/t/tie_ndbm.t   ../ext/GDBM_File/t/fatal.t   ../ext/GDBM_File/t/gdbm.t   ../ext/NDBM_File/t/ndbm.t   ../ext/ODBM_File/t/odbm.t   ../lib/AnyDBM_File.t   op/coreamp.t   op/dbm.t .... u=12.15 s=6.23 cu=402.92 cs=75.09 scripts=2254 tests=674295 make​: *** [test] Error 1 Bad status "2" on exit Make NOT OKAY!

---

Complete log attached in makeperl-26754.log

[p5pRT]

From perl-diddler@tlinx.org

myconfig

[p5pRT]

From perl-diddler@tlinx.org

makeperl-26754.log

[p5pRT]

From @doughera88

On Sun Sep 01 17​:41​:19 2013\, LAWalsh wrote​:

On Sun Sep 01 15​:16​:33 2013\, doughera wrote​:

First\, you provide us with the *minimal* Configure command line necessary to reproduce the problem in a clean perl directory. ---- Going from tar and Configure\, the problem I see now has to do with the DB feature; the dir where the source is built is recreated at the start of my make.

Ok. This is certainly different from the problem originally reported (prototype of syscall). Let's go forward from here.

I had been using Configure -da\, but Merijn suggested​: Configure -Duse64bitall -des\, so altered my script to call that.

Good. (On amd64\, -Duse64bitall doesn't end up doing anything\, but it's harmless.)

Second\, you provide the output of the ./myconfig script from that directory. ---- myconfig script attached.

Ok. That looks pretty normal.

Third\, you provide the exact error messages or test failures that result. ---

Among others​:

lib/AnyDBM_File ............................................... #
Failed test 'Tie' # at ../lib/AnyDBM_File.t line 25. FAILED at test 1 .... Failed 10 tests out of 2254\, 99.56% okay. ../cpan/Memoize/t/errors.t ../cpan/Memoize/t/tie_gdbm.t ../cpan/Memoize/t/tie_ndbm.t ../ext/GDBM_File/t/fatal.t ../ext/GDBM_File/t/gdbm.t ../ext/NDBM_File/t/ndbm.t ../ext/ODBM_File/t/odbm.t ../lib/AnyDBM_File.t op/coreamp.t op/dbm.t .... u=12.15 s=6.23 cu=402.92 cs=75.09 scripts=2254 tests=674295 make​: *** [test] Error 1 Bad status "2" on exit Make NOT OKAY! --------------- Complete log attached in makeperl-26754.log

Thank you. Now let's focus on the gdbm-related errors. It looks like there might be an issue with your gdbm library. Here are a few things to test​:

1. in the perl-5.18.0 directory\, try simply running

  ./perl -Ilib -MGDBM_File -e 1;

  to make sure you can load the module.

2. If that works\, try a simple test of the module\, e.g. something   like (note -- I don't use GDBM\, so these examples are all just   written from the man pages. Nevertheless\, they seem to work   for me.)

#!./perl use warnings; use strict; use lib 'lib'; use GDBM_File; my (%hash\, %check); my ($key\, $val); my $filename = "tied.gdbm"; tie %hash\, 'GDBM_File'\, $filename\, &GDBM_WRCREAT\, 0644; $hash{'firstkey'} = 'firstvalue'; $hash{'lastkey'} = 'lastvalue'; untie %hash;

tie %check\, 'GDBM_File'\, $filename\, &GDBM_READER\, 0644; while (($key\, $val) = each(%check)) {   print "key = $key\, val = $val\n"; } untie %check; unlink $filename;

3. If either #1 or #2 fails\, you can also check the integrity of your gdbm library from a pure C test\, something like this​:

#include \<stdio.h> #include \<stdlib.h> #include \<unistd.h> #include \<string.h> #include \<gdbm.h> #include \<errno.h>

int main (int argc\, char **argv) {   GDBM_FILE dbf;   datum key = { "somekey"\, 8 };   datum value = { "somevalue"\, 8 };   int ret;   datum content;   errno = 0;   printf("gdbm_version = %s\n"\, gdbm_version);   dbf = gdbm_open ("trial.gdbm"\, 0\, GDBM_NEWDB\, 0644\, 0);   if (!dbf) {   printf("%s\, errno = %d\, gdbm_errno = %d\n"\,   "Fatal error opening trial.gdbm"\, errno\, gdbm_errno);   exit(1);   }   ret = gdbm_store (dbf\, key\, value\, GDBM_INSERT);   if (ret != -1) {   printf("Successfully stored key.\n");   }   else {   printf("%s ret = %d\, errno = %d\, gdbm_errno = %d\n"\,   "Failed to store key."\, ret\, errno\, gdbm_errno);   }   content = gdbm_fetch(dbf\, key);   if (content.dptr && strncmp(content.dptr\, value.dptr\, 7) == 0) {   printf("Successfully fetched key.\n");   }   else {   printf("%s errno = %d\, gdbm_errno = %d\n"\,   "Failed to retrieve key."\, errno\, gdbm_errno);   }   gdbm_close(dbf);   unlink("trial.gdbm");   printf ("done.\n");   return 0; }

My suspicion is that there's something wrong with your gdbm installation. If the C test works\, but the perl tests fail\, and they are both using the same libgdbm.so installed shared library\, then we have a real puzzle on our hands. I'm hoping it's something simple.

--   Andy Dougherty doughera@​lafayette.edu

[p5pRT]

From perl-diddler@tlinx.org

On Tue Sep 03 06​:44​:36 2013\, doughera wrote​:

Ok. This is certainly different from the problem originally reported (prototype of syscall). Let's go forward from here.

---

  I think that one might be a case where perl doesn't check or clean ENV vars that might strongly affect how perl generates itself.

  Ideally\, at some point\, it would be useful\, in order to be able to know all of the factors that go into a perl build\, to make sure and ENV vars that could be used on a platform are set to standard or default values while perl is building. Otherwise you end up with something that is difficult to reproduce depending on how you login and where.

But that isn't what's going on here\, at this point.

... shortcutting to the problem that the "C" program you provided very *wonderfully* reproduces. You left the block size @​ "0" in the gdbm_open routing​:   dbf = gdbm_open ("trial.gdbm"\, 0\, GDBM_NEWDB\, 0644\, 0);

  *IF* gdbm was working correctly then this would be a case of using a documented feature. However\, as some of you may remember\, I also tried to get the wording in the perlrun(stat) man page changed to reflect that the blksize returned *wasn't* the size of the blocks returned by the same call.
1st problem is that the gdbm lib\, in the absence of the actual blksize struct member\, uses 1024 as a default for the number of blocks\, which is documented in the manpage as :   blkcnt_t st_blocks; /* number of 512B blocks allocated */

2nd problem) As the code assumes that blksize the size of a block on disk\, (vs. returning suggested I/O size to use when reading/writing to that file on that device).

2a) Somewhere it also makes the assumption that the optimal I/O size will always be a power of 2. For a RAID\, the optimal is the strip-size * the strip-width. On the disk I am doing most of my dev work on\, that's 64k * (12-data groups) or 768K bytes which is NOT a power of 2. That causes failure.

========================== The source of the bug is in gdbm\, last modified in 2011. A workaround for this situation is not leave the block-size field as '0' (asking it to be filled in)\, but set it to a value that will allow the test suite to pass. Values that I've tested in the "C" program you include​: 4K\, 64K\, 512K\, 1M -- what doesn't work​: 768K (the ideal I/O size for the device).

To protect against buggy gdbm (1.10 - 1.8.3 have the problem) implementations\, you might want to specify a record size where you can control that it is a power of 2. Yeah\, we can hope the gdbm maintainers will fix this soon\, and wait for that to percolate out to the distros\, but meanwhile\, IMO\, perl should protect itself so it's build/config process isn't affected by transient bugs (or ENV vars).

My suspicion is that there's something wrong with your gdbm installation.

---

Yup -- and likely nearly 100% of others\, as well\, from 1.8.3-1.10\, which won't trigger until you try to create a DB on a disk subsystem that comes back with a non-power-of-2 strip size. Not common\, but certainly far from impossible.

While I can probably\, minimally\, put in a hack on my machine\, that won't protect anyone else using the feature on a machine that has an ideal 'ioblocksize' that isn't divisible by 2.

This likely explains various problems I've had with Mail​::Spamassassin -- most likely after an upgrade\, where it complains about the data base format and claims it cannot open it.

It's made worse by the rpm that suse installs tries to put spamassassin in some dir under /var and I have to move it to a larger/faster disk (=>/home/spamassassin)\, where that "ioblksize" changes. I didn't know gdbm was sensitive to that optimal IO size\, nor that it might become upset if it was changed on a given DB (due to being relocated to a different disk).

So need to figure out where gdbmopen is called in perl and made sure any values used there are a power of 2\, and in my case\, >=64k (as 1 strip unit = that and that's the minimum size that can be updated on this RAID50.

Hopefully that will let it build & pass testing. But maybe such a workaround belongs in perl as well\, given how long the bug has been in gdbm and no one has run into it?

[p5pRT]

From @doughera88

On Wed\, Sep 04\, 2013 at 09​:57​:46PM -0700\, "Linda Walsh via RT" wrote​:

On Tue Sep 03 06​:44​:36 2013\, doughera wrote​:

My suspicion is that there's something wrong with your gdbm installation.

The source of the bug is in gdbm\, last modified in 2011. A workaround for this situation is not leave the block-size field as '0' (asking it to be filled in)\, but set it to a value that will allow the test suite to pass. Values that I've tested in the "C" program you include​: 4K\, 64K\, 512K\, 1M -- what doesn't work​: 768K (the ideal I/O size for the device).

Good detective work. Thank you.

To protect against buggy gdbm (1.10 - 1.8.3 have the problem) implementations\, you might want to specify a record size where you can control that it is a power of 2. Yeah\, we can hope the gdbm maintainers will fix this soon\, and wait for that to percolate out to the distros\, but meanwhile\, IMO\, perl should protect itself so it's build/config process isn't affected by transient bugs (or ENV vars).

So need to figure out where gdbmopen is called in perl and made sure any values used there are a power of 2\, and in my case\, >=64k (as 1 strip unit = that and that's the minimum size that can be updated on this RAID50.

Hopefully that will let it build & pass testing. But maybe such a workaround belongs in perl as well\, given how long the bug has been in gdbm and no one has run into it?

The quick workaround for you is to edit line 26 in ext/GDBM_FILE/GDBM_FILE.xs which currently reads

#define GDBM_BLOCKSIZE 0 /* gdbm defaults to stat blocksize */

and change the value of '0' to something that will work well for you.

I agree that although the bug is in gdbm\, perl ought to try to work around it\, or at least make it easier for you to work around it. Unfortunately\, we can't just stat() the file\, since it doesn't exist yet. The workaround will have to be a little more involved. I will open up a separate bug in RT for this\, but I don't exepct to get to it anytime very soon.

Meanwhile\, would you be willing to report it to the gdbm maintainers?

Thank you for the clear diagnosis.

--   Andy Dougherty doughera@​lafayette.edu

[p5pRT]

From @doughera88

On Wed\, Sep 04\, 2013 at 09​:57​:46PM -0700\, "Linda Walsh via RT" wrote​:

On Tue Sep 03 06​:44​:36 2013\, doughera wrote​:

Ok. This is certainly different from the problem originally reported (prototype of syscall). Let's go forward from here. ---- I think that one might be a case where perl doesn't check or clean ENV vars that might strongly affect how perl generates itself.

Ideally\, at some point\, it would be useful\, in order to be able

to know all of the factors that go into a perl build\, to make sure and ENV vars that could be used on a platform are set to standard or default values while perl is building. Otherwise you end up with something that is difficult to reproduce depending on how you login and where.

I have filed a bug for the underlying gdbm issue as​: [perl #119623] GDBM_FILE​: gdbm_open requires a blocksize to be a power of two

As for the other build problems\, I don't know what\, specifically\, you are recommending. I don't have any idea which variables you think Configure ought to be setting. Apart from variables related to dynamic loading\, Configure generally doesn't use environment variables to communicate build information.

More broadly\, my general bias is that Configure ought to respect what ever environment variables you set up\, in order to allow you to set them to whatever you want for whatever reason you want. (Configure does alter a few environment variables that have been historically problematic\, but generally we prefer not to.)

I am going to close this ticket. If you have a specific build problem to address\, please open a new ticket with a minimal recipe that someone can use to reproduce the problem.

Thank you\,

--   Andy Dougherty doughera@​lafayette.edu

[p5pRT]

@doughera88 - Status changed from 'open' to 'resolved'

[p5pRT]

From perl-diddler@tlinx.org

On Thu Sep 05 07​:30​:11 2013\, doughera wrote​:

I have filed a bug for the underlying gdbm issue as​: [perl #119623] GDBM_FILE​: gdbm_open requires a blocksize to be a power of two

---

Thanks. Unfortunately\, that only gets around part of the problem.

The other part involves "NDBM/ODBM" -- they both end up calling gdbm_open\, and both end up creating their own value for the blocksize and guess what they use! :-(.... "0"...

On the bright side\, patching that 0 in the gdbm(compat) source to (I used 4k)\, worked -- 5.18 built & tested. Without that part\, the gdbm tests past\, but the ndbm/odbm ones failed.

As for the other build problems\, I don't know what\, specifically\, you are recommending. I don't have any idea which variables you think Configure ought to be setting. Apart from variables related to dynamic loading\, Configure generally doesn't use environment variables to communicate build information.

---

Ok\, let me rephrase -- Suppose someone thinks they want to build perl as utf8 source. I.e. someone makes one or more changes to include utf8 in perl source used in build & test\, so they add "-Mutf8" to PERL5OPS. Will perl build and test with that set in the PERL5OPTS?

Compare that to to making the linux kernel. If I go into the ".config" and tell it to build the kernel with values that wouldn't work -- the kernel make detects that .config was changed and runs it's "lint-config" equiv on the config file to verify you didn't specify an illegal configuration. If so\, it will autocorrect it usually and then continue -- but the 'illegal option' will be removed from the "build-env" (in this case a config file) before the build starts.

Similarly\, if values in PERL5OPTS (like -Mutf8​::all\, or -Mutf8 or whatever) will cause the build to fail in some unclear way\, then what I was suggesting was that perl "check" it's input parameters and attempt to coerce them into validity. If an ENV var is including modules that shouldn't be included\, then it would remove the offending directives.

It may be the case that some values (-M\) are valid options at build time. Ideally those would be placed on a "white list" -- at least that's one way of addressing the issue. I don't know how large the white list would be -- (and this gets a bit squirrelly) or if that is practical.

To augment the default white-list\, or\, to skip extra validation of inputs from ENV & config files\, perhaps an "--as-is" (or whatever!) switch would be a way to re-allow the current abilities to include anything -- with the difference from the current situation\, being that a perl-maker/builder would have to explicitly set "--as-is"\, to use what might otherwise be "bogus" values in their env or input configuration.

More broadly\, my general bias is that Configure ought to respect what ever environment variables you set up\, in order to allow you to set them to whatever you want for whatever reason you want. (Configure does alter a few environment variables that have been historically problematic\, but generally we prefer not to.)

---

  Generally\, I agree -- I'm just thinking of a sanity check to make sure that options that are known to cause problems with the build are filtered. If you think something like -Mutf8 or -Mutf8​::all in your PERL5OPTS is harmless and should be allowed\, might I suggest trying a build with those set? I.e. does it make sense for any reason allow options that might cause a build to fail in some obscure way?

---

As for the gdbm bug submission​:... I sent the below\, yesterday\, but have gotten no response. It may be I'll have to figure out a way to patch the source\, since the default value of '0' used in ndbm/odbm is still a problem even if the gdbm values are fixed. ;-(

---

-------- Original Message -------- Subject​: bug in 1.10​: "optimal i/o size is *assumed* to be power of 2. This is not always true. Date​: Wed\, 04 Sep 2013 23​:09​:35 -0700 From​: Linda A. Walsh \<gnu...tlinx.org> To​: bug-gdbm@​gnu.org

In gdbmopen.c\, ~ lines 235-242 we can see the dir size starting off with 8 'datums' of size (off_t)\, which it says take 3 bits to store. Fine. Then it loops on dir_size \< block_size and uses a leftshift on size\, and +1 on bits until dir_size >= blocksize.

Using a 12-data disk RAID of 64KB/segment\, => 768K = 1 fullwidth stripe on the RAID -- which is exactly what is returned from "stat" when asked for the blocksize.

When dirsize becomes > 768K\, it will have jumped from 512K->1M.

Following that at line 244\, is a check​:

/* Check for correct block_size. */ if (dbf->header->dir_size != dbf->header->block_size)   {   gdbm_close (dbf);   gdbm_errno = GDBM_BLOCK_SIZE_ERROR;   return NULL;   }

---

But dir block size Cannot be equal to the desired block size\, to the way it is calculated by powers of 2.

Either the prog needs to use dir_size/(sizeof(off_t)) to get dir entries\, or if power of 2 is needed for other reasons\, then 256K needs to be "allocated" to padding after the dir_block_size gets to 512K -- thus causing further DB writes to be aligned.

The above was detected using the SuSE factory source rpm for what will be "13.1" of opensuse. Note -- it is also the case that because of this bug\, perl won't build and pass it's DB tests on some machines (like mine) because of this error.

---

Also\, a bit of oddness -- I know that several places use a hard-coded '31' as top end for number of bits.. Might this cause problems on 64-bit machines? (bucket.c being the main offender)

[p5pRT]

From @doughera88

On Thu\, Sep 05\, 2013 at 04​:56​:34PM -0700\, "Linda Walsh via RT" wrote​:

On Thu Sep 05 07​:30​:11 2013\, doughera wrote​:

I have filed a bug for the underlying gdbm issue as​: [perl #119623] GDBM_FILE​: gdbm_open requires a blocksize to be a power of two ---- Thanks. Unfortunately\, that only gets around part of the problem.

The other part involves "NDBM/ODBM" -- they both end up calling gdbm_open\, and both end up creating their own value for the blocksize and guess what they use! :-(.... "0"...

Oh\, right. You're using the gdbm emulations.

On the bright side\, patching that 0 in the gdbm(compat) source to (I used 4k)\, worked -- 5.18 built & tested. Without that part\, the gdbm tests past\, but the ndbm/odbm ones failed.

Fine. Or\, you could just not include ndbm and odbm\, unless you really want them.

As for the other build problems\, I don't know what\, specifically\, you are recommending. I don't have any idea which variables you think Configure ought to be setting. Apart from variables related to dynamic loading\, Configure generally doesn't use environment variables to communicate build information. ---- Ok\, let me rephrase -- Suppose someone thinks they want to build perl as utf8 source. I.e. someone makes one or more changes to include utf8 in perl source used in build & test\, so they add "-Mutf8" to PERL5OPS. Will perl build and test with that set in the PERL5OPTS?

Oh\, ok. That's a different question. Your original problem report never built perl at all\, so all the PERL5* environment variables were completely irrelevant.

During the build\, yes\, some PERL5OPT settings can cause trouble. See [perl #5844] for more details. But again\, that wasn't what was happening in the case reported here.

--   Andy Dougherty doughera@​lafayette.edu

[p5pRT]

From perl-diddler@tlinx.org

On Thu Sep 05 18​:44​:19 2013\, doughera wrote​:

On Thu\, Sep 05\, 2013 at 04​:56​:34PM -0700\, "Linda Walsh via RT" wrote​:

On Thu Sep 05 07​:30​:11 2013\, doughera wrote​:

I have filed a bug for the underlying gdbm issue as​: [perl #119623] GDBM_FILE​: gdbm_open requires a blocksize to be a power of two ---- Thanks. Unfortunately\, that only gets around part of the problem.

The other part involves "NDBM/ODBM" -- they both end up calling gdbm_open\, and both end up creating their own value for the blocksize and guess what they use! :-(.... "0"...

Oh\, right. You're using the gdbm emulations.

On the bright side\, patching that 0 in the gdbm(compat) source to (I used 4k)\, worked -- 5.18 built & tested. Without that part\, the gdbm tests past\, but the ndbm/odbm ones failed.

Fine. Or\, you could just not include ndbm and odbm\, unless you really want them.

As for the other build problems\, I don't know what\, specifically\, you are recommending. I don't have any idea which variables you think Configure ought to be setting. Apart from variables related to dynamic loading\, Configure generally doesn't use environment variables to communicate build information. ---- Ok\, let me rephrase -- Suppose someone thinks they want to build perl as utf8 source. I.e. someone makes one or more changes to include utf8 in perl source used in build & test\, so they add "-Mutf8" to PERL5OPS. Will perl build and test with that set in the PERL5OPTS?

Oh\, ok. That's a different question. Your original problem report never built perl at all\, so all the PERL5* environment variables were completely irrelevant.

---

The initial build failed -- and I think that might have been due to some ENV var\, like PERL5OPT being set for 'normal' usage\, and not being "unset" for a perl build.

During the build\, yes\, some PERL5OPT settings can cause trouble. See [perl #5844] for more details. But again\, that wasn't what was happening in the case reported here.

---

I'm saying that some of the "variability of symptoms" may have been because of ENV vars -- though that's not the problem with the DB issues.

Recently\, I noted that the settings in the "GREP_OPTIONS" var can contaminate a kernel build tree\, such that when you do subsequent builds (even with GREP_OPTIONS cleared)\, it won't build correctly. You need to do a "make distclean" between builds to get rid of "Environmental contamination"...

Perl is small enough that removing the old tree and and untarring a fresh copy take \<1 second\, so that seems to be the easier option\, so after the first attempt in a pre-existing tree in my rpm-build dir\, I switched to fresh-trees on each attempt -- that cleared up many problems right there (I'd last tried to build 5.18-rpms about 3-4 months ago).

Thanks for the help... now have to figure out what to do with ndbm/odbm...

Me\, personally -- I don't care so much\, but wanting it to be compat with the perl released by my distro\, as I usually drop new perl's in place of older distro perls (with recompiles)...