rschupp / PAR-Packer

(perl) Generate stand-alone executables, perl scripts and PAR files https://metacpan.org/pod/PAR::Packer
Other
50 stars 13 forks source link

ld: library not found for -lgcc_s.10.4 on macOS 10.14 #20

Closed dxdc closed 4 years ago

dxdc commented 5 years ago

Running macOS 10.14.6. I feel like I've compiled this previously, but upgrading to the newest PAR failed. Sharing some other debugging messages.

I think these articles may be related to the problem?

I see plenty of files here, like this one: /usr/local/lib/gcc/9/libgcc_ext.10.4.dylib

Some other output, which may be useful:

$ which ld
/usr/bin/ld
$ ld -v dummy
@(#)PROGRAM:ld  PROJECT:ld64-450.3
BUILD 18:16:53 Apr  5 2019
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
Library search paths:
    /usr/lib
    /usr/local/lib
Framework search paths:
    /Library/Frameworks/
    /System/Library/Frameworks/

(installed with -f)

$ cpan -i -f PAR::Packer
Loading internal logger. Log::Log4perl recommended for better logging
Reading '/Users/myuser/.cpan/Metadata'
  Database was generated on Wed, 14 Aug 2019 22:55:26 GMT
Running install for module 'PAR::Packer'
CPAN: Digest::SHA loaded ok (v5.95)
CPAN: Compress::Zlib loaded ok (v2.068)
Checksum for /Users/myuser/.cpan/sources/authors/id/R/RS/RSCHUPP/PAR-Packer-1.049.tar.gz ok
CPAN: YAML loaded ok (v1.15)
CPAN: CPAN::Meta::Requirements loaded ok (v2.133)
CPAN: Parse::CPAN::Meta loaded ok (v1.4417)
CPAN: CPAN::Meta loaded ok (v2.150005)
CPAN: Module::CoreList loaded ok (v5.20160320)
Configuring R/RS/RSCHUPP/PAR-Packer-1.049.tar.gz with Makefile.PL
ld: library not found for -lgcc_s.10.4
clang: error: linker command failed with exit code 1 (use -v to see invocation)
No compiler found, won't generate 'script/parl!
Checking if your kit is complete...
Looks good
Generating a Unix-style Makefile
Writing Makefile for PAR::Packer
Writing MYMETA.yml and MYMETA.json
  RSCHUPP/PAR-Packer-1.049.tar.gz
  /usr/local/bin/perl Makefile.PL -- OK
Running make for R/RS/RSCHUPP/PAR-Packer-1.049.tar.gz
cp lib/App/Packer/PAR.pm blib/lib/App/Packer/PAR.pm
cp lib/PAR/Packer.pm blib/lib/PAR/Packer.pm
cp lib/PAR/Filter.pm blib/lib/PAR/Filter.pm
cp lib/PAR/Filter/Obfuscate.pm blib/lib/PAR/Filter/Obfuscate.pm
cp lib/PAR/Filter/Bleach.pm blib/lib/PAR/Filter/Bleach.pm
cp lib/PAR/Filter/PatchContent.pm blib/lib/PAR/Filter/PatchContent.pm
cp lib/PAR/Filter/PodStrip.pm blib/lib/PAR/Filter/PodStrip.pm
cp lib/PAR/Filter/Bytecode.pm blib/lib/PAR/Filter/Bytecode.pm
cp lib/pp.pm blib/lib/pp.pm
cp lib/PAR/StrippedPARL/Base.pm blib/lib/PAR/StrippedPARL/Base.pm
cp script/tkpp blib/script/tkpp
"/usr/local/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/tkpp
cp script/par.pl blib/script/par.pl
"/usr/local/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/par.pl
cp script/pp blib/script/pp
"/usr/local/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/pp
Manifying 3 pod documents
Manifying 10 pod documents
  RSCHUPP/PAR-Packer-1.049.tar.gz
  /usr/bin/make -- OK
CPAN: CPAN::DistnameInfo loaded ok (v0.12)
Running make test for RSCHUPP/PAR-Packer-1.049.tar.gz
PERL_DL_NONLAZY=1 "/usr/local/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
t/00-pod.t ............... skipped: Set environment variable PERL_TEST_POD=1 to test POD
t/10-parl-generation.t ... 1/31 
#   Failed test 'Found the static build of parl in myldr'
#   at t/10-parl-generation.t line 28.
# Looks like you failed 1 test of 31.
t/10-parl-generation.t ... Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/31 subtests 
    (less 27 skipped subtests: 3 okay)
t/20-pp.t ................ skipped: 'parl' not found
t/30-current_exec.t ...... # Please wait
t/30-current_exec.t ...... ok   
t/40-packer_cd_option.t .. ok   
t/80-doublecolon.t ....... ok   
t/85-crt-glob.t .......... ok   
t/90-rt101800.t .......... 1/18 # PAR_TEMP = /var/folders/rv/spd1m5_524j35pnylts_bxvm0000gn/T/ce7aimSWJO/par-64616e69656c6361737069/cache-4796669d20cf6c8138fcf3569ea592a4929c3226
# running /var/folders/rv/spd1m5_524j35pnylts_bxvm0000gn/T/ce7aimSWJO/packed a second time
t/90-rt101800.t .......... 12/18 # running /var/folders/rv/spd1m5_524j35pnylts_bxvm0000gn/T/ce7aimSWJO/packed a third time
t/90-rt101800.t .......... ok     
t/90-rt103861.t .......... ok   
t/90-rt104560.t .......... ok   
t/90-rt104635.t .......... ok   
t/90-rt122949.t .......... skipped: Tests only relevant on Windows
t/90-rt127064.t .......... skipped: Tests only relevant on Windows
t/90-rt129312.t .......... 1/4 
#   Failed test 'successfully ran "/var/folders/rv/spd1m5_524j35pnylts_bxvm0000gn/T/xKDvYqUyJC/packed"'
#   at ./t/utils.pl line 39.
#          got: '65280'
#     expected: '0'

#   Failed test at t/90-rt129312.t line 23.
#          got: ''
#     expected: 'hello, garbage
# '
# stderr: Usage: /var/folders/rv/spd1m5_524j35pnylts_bxvm0000gn/T/xKDvYqUyJC/packed [ -Alib.par ] [ -Idir ] [ -Mmodule ] [ src.par ] [ program.pl ]
# /var/folders/rv/spd1m5_524j35pnylts_bxvm0000gn/T/xKDvYqUyJC/packed [ -B|-b ] [-Ooutfile] src.par
# Looks like you failed 2 tests of 4.
t/90-rt129312.t .......... Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/4 subtests 
t/90-rt59710.t ........... ok   

Test Summary Report
-------------------
t/10-parl-generation.t (Wstat: 256 Tests: 31 Failed: 1)
  Failed test:  4
  Non-zero exit status: 1
t/90-rt129312.t        (Wstat: 512 Tests: 4 Failed: 2)
  Failed tests:  3-4
  Non-zero exit status: 2
Files=15, Tests=83, 16 wallclock secs ( 0.05 usr  0.03 sys +  7.35 cusr  2.24 csys =  9.67 CPU)
Result: FAIL
Failed 2/15 test programs. 3/83 subtests failed.
make: *** [test_dynamic] Error 255
  RSCHUPP/PAR-Packer-1.049.tar.gz
  /usr/bin/make test -- NOT OK
//hint// to see the cpan-testers results for installing this module, try:
  reports RSCHUPP/PAR-Packer-1.049.tar.gz
Running make install for RSCHUPP/PAR-Packer-1.049.tar.gz
Manifying 3 pod documents
Manifying 10 pod documents
Installing /usr/local/lib/perl5/site_perl/5.18.2/pp.pm
Installing /usr/local/lib/perl5/site_perl/5.18.2/PAR/Packer.pm
Installing /usr/local/share/man/man1/tkpp.1
Installing /usr/local/share/man/man1/par.pl.1
Installing /usr/local/share/man/man3/PAR::StrippedPARL::Base.3
Installing /usr/local/share/man/man3/PAR::Packer.3
Installing /usr/local/share/man/man3/PAR::Filter::PatchContent.3
Installing /usr/local/share/man/man3/pp.3
Installing /usr/local/share/man/man3/PAR::Filter::Bytecode.3
Installing /usr/local/share/man/man3/PAR::Filter::PodStrip.3
Installing /usr/local/share/man/man3/PAR::Filter::Bleach.3
Installing /usr/local/share/man/man3/PAR::Filter::Obfuscate.3
Installing /usr/local/share/man/man3/PAR::Filter.3
Installing /usr/local/share/man/man3/App::Packer::PAR.3
Installing /usr/local/bin/par.pl
Appending installation info to /usr/local/lib/perl5/5.18.2/darwin-2level/perllocal.pod
  RSCHUPP/PAR-Packer-1.049.tar.gz
  /usr/bin/make install  -- OK
rschupp commented 5 years ago

Here's the problem:

ld: library not found for -lgcc_s.10.4 clang: error: linker command failed with exit code 1 (use -v to see invocation) No compiler found, won't generate 'script/parl!

These were produced by calling

my $have_cc = ExtUtils::CBuilder->new->have_compiler;

which relies on the information from perl's Config. Please post the output of type perl and perl -V.

dxdc commented 5 years ago

You're right @rschupp !

I dug into this and found some relevant information. The problem is related to perlbrew. I tried on a system install (/usr/bin/cpan) and that worked just fine.

I'm not sure if there's more work to be done at this point to support perlbrew installations, but I appreciate you helping me to at least figure out the root cause ExtUtils::CBuilder. I tried updating ExtUtils::CBuilder with CPAN but that doesn't change the result.

One point that could be relevant, is that I compiled perlbrew to get versions that would be compatible on older platforms like this:

perlbrew install 5.18.4 -Accflags="-mmacosx-version-min=10.7" -Aldflags="-mmacosx-version-min=10.7" -Alddlflags="-mmacosx-version-min=10.7" --notest

Simplified way to reproduce the issue (as you pointed out):

#!/usr/bin/perl
use ExtUtils::CBuilder;
my $have_cc = ExtUtils::CBuilder->new->have_compiler;

Which returns:

ld: warning: directory not found for option '-L/usr/local/opt/lib'
ld: library not found for -lgcc_s.10.4
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Here is the perl -V output:

(Perlbrew install) : /usr/local/bin/perl -V

Summary of my perl5 (revision 5 version 18 subversion 2) configuration:

  Platform:
    osname=darwin, osvers=13.1.0, archname=darwin-2level
    uname='darwin dcaspi.local 13.1.0 darwin kernel version 13.1.0: thu jan 16 19:40:37 pst 2014; root:xnu-2422.90.20~2release_x86_64 x86_64 '
    config_args=''
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=undef, usemultiplicity=undef
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags ='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -I/opt/local/include',
    optimize='-O3',
    cppflags='-fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -I/opt/local/include'
    ccversion='', gccversion='4.2.1 Compatible Apple LLVM 5.1 (clang-503.0.38)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='env MACOSX_DEPLOYMENT_TARGET=10.3 cc', ldflags =' -fstack-protector -L/usr/local/lib -L/opt/local/lib'
    libpth=/usr/local/lib /opt/local/lib /usr/lib
    libs=-ldbm -ldl -lm -lutil -lc
    perllibs=-ldl -lm -lutil -lc
    libc=, so=dylib, useshrplib=false, libperl=libperl.a
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -L/usr/local/lib -L/opt/local/lib -fstack-protector'

Characteristics of this binary (from libperl): 
  Compile-time options: HAS_TIMES PERLIO_LAYERS PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD PERL_MALLOC_WRAP
                        PERL_PRESERVE_IVUV PERL_SAWAMPERSAND USE_64_BIT_ALL
                        USE_64_BIT_INT USE_LARGE_FILES USE_LOCALE
                        USE_LOCALE_COLLATE USE_LOCALE_CTYPE
                        USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF
  Built under darwin
  Compiled at Apr  9 2014 22:20:07
  @INC:
    /usr/local/lib/perl5/site_perl/5.18.2/darwin-2level
    /usr/local/lib/perl5/site_perl/5.18.2
    /usr/local/lib/perl5/5.18.2/darwin-2level
    /usr/local/lib/perl5/5.18.2
    /usr/local/lib/perl5/site_perl
    .

(System Install) : /usr/bin/perl -V

Summary of my perl5 (revision 5 version 18 subversion 4) configuration:

  Platform:
    osname=darwin, osvers=18.0, archname=darwin-thread-multi-2level
    uname='darwin osx316.apple.com 18.0 darwin kernel version 17.0.0: fri may 4 10:33:38 pdt 2018; root:xnu-4570.1.46.100.2~1development_x86_64 x86_64 '
    config_args='-ds -e -Dprefix=/usr -Dccflags=-g  -pipe  -Dldflags= -Dman3ext=3pm -Duseithreads -Duseshrplib -Dinc_version_list=none -Dcc=cc'
    hint=recommended, useposix=true, d_sigaction=define
    useithreads=define, usemultiplicity=define
    useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
    use64bitint=define, use64bitall=define, uselongdouble=undef
    usemymalloc=n, bincompat5005=undef
  Compiler:
    cc='cc', ccflags =' -g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector',
    optimize='-Os',
    cppflags='-g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector'
    ccversion='', gccversion='4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)', gccosandvers=''
    intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
    d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
    ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t', lseeksize=8
    alignbytes=8, prototype=define
  Linker and Libraries:
    ld='cc', ldflags =' -fstack-protector'
    libpth=/usr/lib /usr/local/lib
    libs= 
    perllibs=
    libc=, so=dylib, useshrplib=true, libperl=libperl.dylib
    gnulibc_version=''
  Dynamic Linking:
    dlsrc=dl_dlopen.xs, dlext=bundle, d_dlsymun=undef, ccdlflags=' '
    cccdlflags=' ', lddlflags=' -bundle -undefined dynamic_lookup -fstack-protector'

Characteristics of this binary (from libperl): 
  Compile-time options: HAS_TIMES MULTIPLICITY PERLIO_LAYERS
                        PERL_DONT_CREATE_GVSV
                        PERL_HASH_FUNC_ONE_AT_A_TIME_HARD
                        PERL_IMPLICIT_CONTEXT PERL_MALLOC_WRAP
                        PERL_PRESERVE_IVUV PERL_SAWAMPERSAND USE_64_BIT_ALL
                        USE_64_BIT_INT USE_ITHREADS USE_LARGE_FILES
                        USE_LOCALE USE_LOCALE_COLLATE USE_LOCALE_CTYPE
                        USE_LOCALE_NUMERIC USE_PERLIO USE_PERL_ATOF
                        USE_REENTRANT_API
  Locally applied patches:
    /Library/Perl/Updates/<version> comes before system perl directories
    installprivlib and installarchlib points to the Updates directory
  Built under darwin
  Compiled at Apr  1 2019 13:12:58
  @INC:
    /Library/Perl/5.18/darwin-thread-multi-2level
    /Library/Perl/5.18
    /Network/Library/Perl/5.18/darwin-thread-multi-2level
    /Network/Library/Perl/5.18
    /Library/Perl/Updates/5.18.4
    /System/Library/Perl/5.18/darwin-thread-multi-2level
    /System/Library/Perl/5.18
    /System/Library/Perl/Extras/5.18/darwin-thread-multi-2level
    /System/Library/Perl/Extras/5.18
    .
rschupp commented 5 years ago

Hmm, your perlbrew perl seems to have been built with another version of clang than the system version (or maybe env MACOSX_DEPLOYMENT_TARGET=10.3 cc selects a different version).

Comment out the line (~215)

    local $self->{quiet} = 1;

in .../ExtUtils/CBuilder/Base.pm and run (try both with the system and perlbrew perl)

perl -MExtUtils::CBuilder 'say ExtUtils::CBuilder->new->have_compiler'

then you should see the exact commands that ExtUtils::CBuilder executes.

dxdc commented 5 years ago

I think it should have been:

perl -MExtUtils::CBuilder -E 'CORE::say ExtUtils::CBuilder->new->have_compiler'

Perlbrew Perl:

cc -I/usr/local/lib/perl5/5.18.2/darwin-2level/CORE -c -fno-common -DPERL_DARWIN -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -I/opt/local/include -O3 -o compilet-5Q0ps.o compilet-5Q0ps.c
env MACOSX_DEPLOYMENT_TARGET=10.3 cc -bundle -undefined dynamic_lookup -L/usr/local/lib -L/opt/local/lib -fstack-protector -o compilet-5Q0ps.bundle compilet-5Q0ps.o
ld: library not found for -lgcc_s.10.4
clang: error: linker command failed with exit code 1 (use -v to see invocation)
0

System Perl:

cc -I/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE -c -g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector -Os -o compilet-vZUeS.o compilet-vZUeS.c
env LD_RUN_PATH=/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE cc -bundle -undefined dynamic_lookup -fstack-protector -o compilet-vZUeS.bundle compilet-vZUeS.o
1
rschupp commented 5 years ago

Interesting - where does

env LD_RUN_PATH=/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE cc ...

come from, it doesn't show in perl -V?

dxdc commented 5 years ago

Likely here:

CBuilder/Base.pm:  warn "PERL_CORE is set but I can't find your perl source!\n";
CBuilder/Base.pm:  return ''; # return empty string if $ENV{PERL_CORE} but can't find dir ???
CBuilder/Base.pm:  $self->perl_src() || File::Spec->catdir($self->{config}{archlibexp},"CORE");
rschupp commented 5 years ago

More likely in ...ExtUtils/CBuilder/Platform/Unix.pm (look for 'env'). There's also .../ExtUtils/CBuilder/Platform/darwin.pm.

dxdc commented 5 years ago

Just confirmed it comes from that line in Base.pm. I tried replacing CORE with CORE_TEST and sure enough, it changed the output.