search

nodejs / build

Better build and test infra for Node.
504 stars 165 forks source link

AIX: gcc upgrade requirement #925

Closed gibfahn closed 4 years ago

gibfahn commented 6 years ago

As a result of https://github.com/nodejs/node/issues/14785 and https://github.com/nodejs/help/issues/784 we have discovered that a backwards incompatible change was made in libstdc++ AIX between gcc 4.8 and gcc 4.9. This means that the requirement for GCC 4.9.3 for Node 8 requires that we raise the minimum gcc/libstdc++ version for that platform. Ideally we should do this before Node 8 goes LTS, or stick with gcc 4.8.4 for 8.x.

AIX is a special case here, because it doesn't come with libstdc++ or gcc by default, so everyone running on the platform needs to install gcc in order to run Node. There are two places to get gcc versions, the AIX Toolbox and Bull Freeware. Currently the AIX toolbox only has GCC 6.3.0 available, and Bull Freeware has 7.2.0, 6.4.0, and 5.5.0 (and older minor versions).

It's worth noting at this point that gcc changed their versioning scheme after 4.9, so 4.10 became 5.0, and 4.11 became 6.0. So the gap between 4.9 and 7.0 is not as large as it seems.

Given that AIX users have to download gcc anyway, and given that they will have to update their gcc levels for this release, I think we should bump to a relatively new version (so they don't have to upgrade again for as long as possible). I'd suggest using 6.3.0, GCC 7 came out this year, so might be too new to move to.

mhdawson commented 6 years ago

We'll need to have both the existing version of gcc (to build older node versions) as well as the newer version and figure out how to make them co-exist.

I'm +1 for going with 6.3.0

seishun commented 6 years ago

@gibfahn Any progress on this? Any way I can help?

We'll need to have both the existing version of gcc (to build older node versions) as well as the newer version and figure out how to make them co-exist.

Is it not possible to run node built with gcc 4.9 on AIX with gcc 4.8 installed?

gibfahn commented 6 years ago

Is it not possible to run node built with gcc 4.9 on AIX with gcc 4.8 installed?

Unfortunately not, you need the right version of libstdc++.

seawatcher2011 commented 6 years ago

I am receiving the same error on AIX 7.1 while while verifying node ibm-8.9.4.0-node-v8.9.4-aix-ppc64.bin installation. Is there now a fix for the problem? Thanks.

oracle@stpisa02sa:/opt/freeware/lib/pthread/ppc64/ [] node --version exec(): 0509-036 Cannot load program node because of the following errors: 0509-130 Symbol resolution failed for node because: 0509-136 Symbol _ZNKSt9typeinfoeqERKS (number 359) is not exported from dependent module /opt/freeware/lib/pthread/ppc64/libstdc++.a[libstdc++.so.6]. 0509-192 Examine .loader section symbols with the 'dump -Tv' command.

gibfahn commented 6 years ago

@seawatcher2011 do you have gcc 4.8 installed? See https://github.com/nodejs/node/issues/14785 for more info.

seawatcher2011 commented 6 years ago

@gibfahn - no, not gcc 4.8. I have gcc 6.3.0. What is the suggestion workaround? Deinstall gcc6.3.0 and install gcc 4.8?

oracle@stpisa02sa:/opt/freeware/lib/pthread/ppc64/ []  gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/freeware/libexec/gcc/powerpc-ibm-aix7.1.0.0/6.3.0/lto-wrapper
Target: powerpc-ibm-aix7.1.0.0
Configured with: ../gcc-6.3.0/configure --prefix=/opt/freeware --mandir=/opt/freeware/man --infodir=/opt/freeware/info --with-local-prefix=/opt/freeware --with-as=/usr/bin/as --with-ld=/usr/bin/ld --enable-languages=c,c++,fortran,objc,obj-c++ --enable-version-specific-runtime-libs --disable-nls --enable-decimal-float=dpd --with-cloog=no --with-ppl=no --disable-libstdcxx-pch --enable-__cxa_atexit --host=powerpc-ibm-aix7.1.0.0
Thread model: aix
gcc version 6.3.0 (GCC)
oracle@stpisa02sa:/opt/freeware/lib/pthread/ppc64/ [] lslpp -Lq | grep c++
  libstdc++                  6.3.0-1    C     R    GNU Standard C++ Library
oracle@stpisa02sa:/opt/freeware/lib/pthread/ppc64/ []  lslpp -Lq | grep gcc
  gcc                        6.3.0-1    C     R    GNU Compiler Collection
  gcc-cpp                    6.3.0-1    C     R    The C Preprocessor (/bin/rpm)
  libgcc                     6.3.0-1    C     R    GCC version 6.3.0 shared
gibfahn commented 6 years ago

@gibfahn - no, not gcc 4.8. I have gcc 6.3.0. What is the suggestion workaround? Deinstall gcc6.3.0 and install gcc 4.8?

Unfortunately you will need 4.8. However you should be able to have them both installed side-by-side. I've heard this is possible but haven't confirmed it myself, that's in fact the point of this issue.

gibfahn commented 6 years ago

I have now managed to get this working, but there was one pain point.

On one machine I tested on I got this error persistently, implying that the LIBPATH wasn't being passed down to the g++ call.

  LD_LIBRARY_PATH=/home/gib/node/out/Release/lib.host:/home/gib/node/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../.; mkdir -p /home/gib/node/out/Release/obj/gen; python tools/js2c.py "/home/gib/node/out/Release/obj/gen/node_javascript.cc" lib/internal/bootstrap/loaders.js lib/internal/bootstrap/node.js lib/async_hooks.js lib/assert.js lib/buffer.js lib/child_process.js lib/console.js lib/constants.js lib/crypto.js lib/cluster.js lib/dgram.js lib/dns.js lib/domain.js lib/events.js lib/fs.js lib/fs/promises.js lib/http.js lib/http2.js lib/_http_agent.js lib/_http_client.js lib/_http_common.js lib/_http_incoming.js lib/_http_outgoing.js lib/_http_server.js lib/https.js lib/inspector.js lib/module.js lib/net.js lib/os.js lib/path.js lib/perf_hooks.js lib/process.js lib/punycode.js lib/querystring.js lib/readline.js lib/repl.js lib/stream.js lib/_stream_readable.js lib/_stream_writable.js lib/_stream_duplex.js lib/_stream_transform.js lib/_stream_passthrough.js lib/_stream_wrap.js lib/string_decoder.js lib/sys.js lib/timers.js lib/tls.js lib/_tls_common.js lib/_tls_wrap.js lib/tty.js lib/url.js lib/util.js lib/v8.js lib/vm.js lib/zlib.js lib/internal/async_hooks.js lib/internal/buffer.js lib/internal/cli_table.js lib/internal/child_process.js lib/internal/cluster/child.js lib/internal/cluster/master.js lib/internal/cluster/round_robin_handle.js lib/internal/cluster/shared_handle.js lib/internal/cluster/utils.js lib/internal/cluster/worker.js lib/internal/crypto/certificate.js lib/internal/crypto/cipher.js lib/internal/crypto/diffiehellman.js lib/internal/crypto/hash.js lib/internal/crypto/pbkdf2.js lib/internal/crypto/random.js lib/internal/crypto/sig.js lib/internal/crypto/util.js lib/internal/constants.js lib/internal/encoding.js lib/internal/errors.js lib/internal/freelist.js lib/internal/fs.js lib/internal/http.js lib/internal/inspector_async_hook.js lib/internal/linkedlist.js lib/internal/modules/cjs/helpers.js lib/internal/modules/cjs/loader.js lib/internal/modules/esm/loader.js lib/internal/modules/esm/create_dynamic_module.js lib/internal/modules/esm/default_resolve.js lib/internal/modules/esm/module_job.js lib/internal/modules/esm/module_map.js lib/internal/modules/esm/translators.js lib/internal/safe_globals.js lib/internal/net.js lib/internal/os.js lib/internal/process/esm_loader.js lib/internal/process/next_tick.js lib/internal/process/promises.js lib/internal/process/stdio.js lib/internal/process/warning.js lib/internal/process.js lib/internal/querystring.js lib/internal/process/write-coverage.js lib/internal/readline.js lib/internal/repl.js lib/internal/repl/await.js lib/internal/socket_list.js lib/internal/test/binding.js lib/internal/test/unicode.js lib/internal/timers.js lib/internal/tls.js lib/internal/trace_events_async_hooks.js lib/internal/tty.js lib/internal/url.js lib/internal/util.js lib/internal/util/comparisons.js lib/internal/util/inspector.js lib/internal/util/types.js lib/internal/http2/core.js lib/internal/http2/compat.js lib/internal/http2/util.js lib/internal/v8.js lib/internal/v8_prof_polyfill.js lib/internal/v8_prof_processor.js lib/internal/stream_base_commons.js lib/internal/vm/module.js lib/internal/streams/lazy_transform.js lib/internal/streams/async_iterator.js lib/internal/streams/buffer_list.js lib/internal/streams/duplexpair.js lib/internal/streams/legacy.js lib/internal/streams/destroy.js lib/internal/streams/state.js lib/internal/wrap_js_stream.js deps/v8/tools/splaytree.js deps/v8/tools/codemap.js deps/v8/tools/consarray.js deps/v8/tools/csvparser.js deps/v8/tools/profile.js deps/v8/tools/profile_view.js deps/v8/tools/logreader.js deps/v8/tools/arguments.js deps/v8/tools/tickprocessor.js deps/v8/tools/SourceMap.js deps/v8/tools/tickprocessor-driver.js deps/node-inspect/lib/_inspect.js deps/node-inspect/lib/internal/inspect_client.js deps/node-inspect/lib/internal/inspect_repl.js deps/acorn/dist/acorn.js deps/acorn/dist/walk.js ./config.gypi tools/check_macros.py src/notrace_macros.py src/noperfctr_macros.py tools/nodcheck_macros.py
  touch e54431f42526df1774c55ade864265c11db54855.intermediate
  LD_LIBRARY_PATH=/home/gib/node/out/Release/lib.host:/home/gib/node/out/Release/lib.target:$LD_LIBRARY_PATH; export LD_LIBRARY_PATH; cd ../deps/v8/gypfiles; mkdir -p /home/gib/node/out/Release/obj/gen/src/inspector/protocol /home/gib/node/out/Release/obj/gen/include/inspector; python ../third_party/inspector_protocol/CodeGenerator.py --jinja_dir ../third_party --output_base "/home/gib/node/out/Release/obj/gen/src/inspector" --config ../src/inspector/inspector_protocol_config.json
  /home/gib/tmp/gcc-6.3.0-1/opt/freeware/bin/g++ '-DU_COMMON_IMPLEMENTATION=1' '-DU_ATTRIBUTE_DEPRECATED=' '-D_CRT_SECURE_NO_DEPRECATE=' '-DU_STATIC_IMPLEMENTATION=1' '-DUCONFIG_NO_SERVICE=1' '-DUCONFIG_NO_REGULAR_EXPRESSIONS=1' '-DU_ENABLE_DYLOAD=0' '-DU_HAVE_STD_STRING=1' '-DUCONFIG_NO_BREAK_ITERATION=0' -I../deps/icu-small/source/common  -pthread -Wall -Wextra -Wno-unused-parameter -maix64 -Wno-deprecated-declarations -O3 -fno-omit-frame-pointer -fno-exceptions -std=gnu++1y -frtti -MMD -MF /home/gib/node/out/Release/.deps//home/gib/node/out/Release/obj.target/icuucx/deps/icu-small/source/common/appendable.o.d.raw   -c -o /home/gib/node/out/Release/obj.target/icuucx/deps/icu-small/source/common/appendable.o ../deps/icu-small/source/common/appendable.cpp
exec(): 0509-036 Cannot load program /home/gib/tmp/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1plus because of the following errors:
    0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
    0509-152   Member libmpc.so.3 is not found in archive 
gmake[1]: *** [/home/gib/node/out/Release/obj.target/icuucx/deps/icu-small/source/common/appendable.o] Error 1
gmake[1]: Leaving directory `/home/gib/node/out'
gmake: *** [node] Error 2

With @refack's help I eventually tracked this down to this mailing list discussion: https://lists.gnu.org/archive/html/bug-make/2001-03/msg00067.html

$ ls -l `which gmake`
lrwxrwxrwx    1 root     system           27 Jul 25 2013  /usr/bin/gmake -> ../../opt/freeware/bin/make

$ ls -l /opt/freeware/bin/make
-rwxr-sr-x    1 root     system       280288 Sep 22 2011  /opt/freeware/bin/make

Running sudo chmod 755 /opt/freeware/bin/make did indeed resolve the issue.

The setuid bit is not set in the test or release machines (although it was on the machine I was using to test).

gibfahn commented 6 years ago

@seawatcher2011 your mileage may vary, but what worked for me is to extract the contents of the libstdc++ rpm into a directory, then set the LIBPATH to point to that directory.

I used rpm2cpio to extract the rpm, which isn't available on AIX (I got it from a RHEL box, but I think it's available on Ubuntu).

So first you need to download the rpm from wherever you got gcc from, either AIX Toolbox or Bull Freeware. Either should work, but you can't mix and match.

AIX Toolbox: ftp://public.dhe.ibm.com/aix/freeSoftware/aixtoolbox/RPMS/ppc-6.1/gcc/libstdcplusplus-4.8.5-1.aix6.1.ppc.rpm Bull Freeware: I can't get a direct link, but for AIX 6.1 the file is libstdc++-4.8.4-6.aix6.1.ppc.rpm

You extract the file (on a different machine) with:

rpm2cpio libstdcplusplus-4.8.5-1.aix6.1.ppc.rpm | cpio -idmv

Then zip it up and scp it to the machine. Unzip it, and then add the opt/freeware/lib/pthread/ppc64 subdirectory to your LIBPATH

export LIBPATH="$PWD/opt/freeware/lib/pthread/ppc64:$LIBPATH"

You should then be able to do ldd ./node and ./node and it should all work.

Note that Node 10 is expected to be built with gcc 6.3, so you won't need this workaround once you upgrade to that version.

For info, the reason this is necessary is because there was a breaking change in libstdc++ on AIX between gcc 4.8 and 4.9.

Let me know if it works!

gibfahn commented 6 years ago

I have generated a zip file containing gcc and dependencies, by downloading the zip from Bull Freeware. For some reason the dependency resolver on Bull Freeware downloaded the wrong version of libstdc++ (7.1 instead of 6.1) so I had to manually fix up the with-deps zip.

#!/bin/bash -ex

# Builds a zip file containing gcc and dependencies, installed from Bull Freeware rpms.

rm -rf rpms gcc-6.3.0-1
rm -f gcc-6.3.0-1.zip
mkdir rpms gcc-6.3.0-1

cp gcc-c++-6.3.0-1.aix6.1.ppc.rpm-with-deps.zip rpms
(cd rpms && unzip gcc-c++-6.3.0-1.aix6.1.ppc.rpm-with-deps.zip)

for rpm in rpms/*.rpm; do
  (cd gcc-6.3.0-1 && rpm2cpio ../$rpm | cpio -idmv)
done

zip -r gcc-6.3.0-1.zip gcc-6.3.0-1/

Once you have the zip on the system, it's just a matter of unzipping it somewhere, and then setting up the paths correctly, so:

# Setting the LIBPATH doesn't seem to be necessary.
# export LIBPATH="$PWD/gcc-6.3.0-1/opt/freeware/lib/pthread/ppc64:$PWD/gcc-6.3.0-1/opt/freeware/lib"
export PATH="$PWD/gcc-6.3.0-1/opt/freeware/bin:$PATH"
cd ~/node
export CC=`which gcc` CXX=`which g++` CXX_host=`which g++`
./configure --dest-cpu=ppc64 && gmake -j4

The version I generated is stored at https://ci.nodejs.org/downloads/aix/ (see https://github.com/nodejs/build/issues/871#issuecomment-329952135), there's a zip and an md5sum for checking that everything downloads correctly.

gibfahn commented 6 years ago

I also ran with C++ 14 enabled to make sure that worked, by applying this diff:

diff --git a/common.gypi b/common.gypi
index 6d83d8a..dcbb2a0 100644
--- a/common.gypi
+++ b/common.gypi
@@ -321,7 +321,7 @@
       }],
       [ 'OS in "linux freebsd openbsd solaris android aix cloudabi"', {
         'cflags': [ '-Wall', '-Wextra', '-Wno-unused-parameter', ],
-        'cflags_cc': [ '-fno-rtti', '-fno-exceptions', '-std=gnu++1y' ],
+        'cflags_cc': [ '-fno-rtti', '-fno-exceptions', '-std=gnu++14' ],
         'ldflags': [ '-rdynamic' ],
         'target_conditions': [
           # The 1990s toolchain on SmartOS can't handle thin archives.
seishun commented 6 years ago

'-std=gnu++1y' and '-std=gnu++14' are synonyms.

gibfahn commented 6 years ago

'-std=gnu++1y' and '-std=gnu++14' are synonyms.

Ahh, so how would I enable gcc 14 features (to check that it works)? I want to get something that fails on 4.8.4 and passes on 6.3.0.

seishun commented 6 years ago

You could try to build this PR: https://github.com/nodejs/node/pull/13676

mhdawson commented 6 years ago

Other platforms seem to need to export LDFLAGS, sounds like that is not required on AIX for your testing?

mhdawson commented 6 years ago

Seems to fail on community machine with:

compiler set to 6.3
+ gmake run-ci -j 5
+ NODE_TEST_DIR=/home/iojs/node-tmp PYTHON=python FLAKY_TESTS=dontcare CONFIG_FLAGS= --dest-cpu=ppc64
python ./configure  --dest-cpu=ppc64
exec(): 0509-036 Cannot load program /home/iojs/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1plus because of the following errors:
    0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
    0509-152   Member libmpc.so.3 is not found in archive 
exec(): 0509-036 Cannot load program /home/iojs/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1 because of the following errors:
    0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
    0509-152   Member libmpc.so.3 is not found in archive 
WARNING: C++ compiler too old, need g++ 4.9.4 or clang++ 3.4.2 (CXX=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++)
WARNING: C compiler too old, need gcc 4.2 or clang 3.2 (CC=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc)

Will try to set LIBPATH

mhdawson commented 6 years ago

LIBPATH does not seem to help and just causes other problems.

mhdawson commented 6 years ago

Despite it not looking like the bit was set on the test machine, this was still required:

sudo chmod 755 /opt/freeware/bin/make

mhdawson commented 6 years ago

That seems to start building(That was incorrect, was running with old compiler, fails locally as well) when run locally through the console,but still fails when run through the CI with:

python ./configure  --dest-cpu=ppc64
exec(): 0509-036 Cannot load program /home/iojs/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1plus because of the following errors:
    0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
    0509-152   Member libmpc.so.3 is not found in archive 
exec(): 0509-036 Cannot load program /home/iojs/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1 because of the following errors:
    0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
    0509-152   Member libmpc.so.3 is not found in archive
refack commented 6 years ago

@mhdawson watch out for the following in the Jenkins config... image

BTW: I've found that the following enables building (and apparently the order in important):

export LIBPATH=/opt/freeware/lib:/home/iojs/gcc-6.3.0-1/opt/freeware/lib
mhdawson commented 6 years ago

@refack Thanks ! trying that now.

mhdawson commented 6 years ago

Still failing in CI

Setting compiler for Node version 10 on AIX
+ [ 10 -gt 9 ]
+ export LIBPATH=/opt/freeware/lib:/home/iojs/gcc-6.3.0-1/opt/freeware/lib
+ export PATH=/home/iojs/gcc-6.3.0-1/opt/freeware/bin:/opt/freeware/bin/ccache:/home/iojs/jdk8/jre/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin
+ which gcc
+ which g++
+ which g++
+ export CC=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc CXX=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++ CXX_host=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++
+ echo Compiler set to 6.3
Compiler set to 6.3
+ gmake run-ci -j 5
+ NODE_TEST_DIR=/home/iojs/node-tmp PYTHON=python FLAKY_TESTS=dontcare CONFIG_FLAGS= --dest-cpu=ppc64
python ./configure  --dest-cpu=ppc64
exec(): 0509-036 Cannot load program /home/iojs/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1plus because of the following errors:
    0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
    0509-152   Member libmpc.so.3 is not found in archive 
exec(): 0509-036 Cannot load program /home/iojs/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1 because of the following errors:
    0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
    0509-152   Member libmpc.so.3 is not found in archive 
WARNING: C++ compiler too old, need g++ 4.9.4 or clang++ 3.4.2 (CXX=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++)
WARNING: C compiler too old, need gcc 4.2 or clang 3.2 (CC=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc)

probably unrelated but the compiler detection is also failing:

WARNING: C++ compiler too old, need g++ 4.9.4 or clang++ 3.4.2 (CXX=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++)
WARNING: C compiler too old, need gcc 4.2 or clang 3.2 (CC=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc)
refack commented 6 years ago

So it's more delicate: GCC need LIBPATH to be export LIBPATH=/home/iojs/gcc-6.3.0-1/opt/freeware/lib:/opt/freeware/lib

-bash-4.3$ export LIBPATH=/home/iojs/gcc-6.3.0-1/opt/freeware/lib:/opt/freeware/lib
-bash-4.3$ ./configure --dest-cpu=ppc64
WARNING: openssl_no_asm is enabled due to missed or old assembler.
            Please refer BUILDING.md
creating icu_config.gypi
* Using ICU in deps/icu-small
creating icu_config.gypi

gmake need it the other way:

-bash-4.3$ gmake
exec(): 0509-036 Cannot load program gmake because of the following errors:
        0509-022 Cannot load module /opt/freeware/lib/libintl.a(libintl.so.1).
        0509-150   Dependent module /home/iojs/gcc-6.3.0-1/opt/freeware/lib/libiconv.a(shr4.o) could not be loaded.
        0509-152   Member shr4.o is not found in archive
        0509-022 Cannot load module make.
        0509-150   Dependent module /opt/freeware/lib/libintl.a(libintl.so.1) could not be loaded.
        0509-022 Cannot load module .
-bash-4.3$ export LIBPATH=/opt/freeware/lib:/home/iojs/gcc-6.3.0-1/opt/freeware/lib
-bash-4.3$ gmake
gmake -C out BUILDTYPE=Release V=1
gmake[1]: Entering directory `/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/out'
node.target.mk:13: warning: overriding recipe for target `/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/out/Release/node.exp'
cctest.target.mk:13: warning: ignoring old recipe for target `/home/iojs/build/workspace/node-test-commit-aix/nodes/aix61-ppc64/out/Release/node.exp'
♥gmake: *** [node] Interrupt
mhdawson commented 6 years ago

Looks to me that there is a circular issue, gmake requires LIBPATH in particular order and CC in a different order:

bash-4.3$ sh doit.sh
exec(): 0509-036 Cannot load program gmake because of the following errors:
        0509-022 Cannot load module /opt/freeware/lib/libintl.a(libintl.so.1).
        0509-150   Dependent module /home/iojs/gcc-6.3.0-1/opt/freeware/lib/libiconv.a(shr4.o) could not be loaded.
        0509-152   Member shr4.o is not found in archive 
        0509-022 Cannot load module make.
        0509-150   Dependent module /opt/freeware/lib/libintl.a(libintl.so.1) could not be loaded.
        0509-022 Cannot load module .
bash-4.3$ export LIBPATH=/opt/freeware/lib/home/iojs/gcc-6.3.0-1/opt/freeware/lib                        
bash-4.3$ export LIBPATH=/opt/freeware/lib:/opt/freeware/lib/home/iojs/gcc-6.3.0-1/opt/freeware/lib
bash-4.3$ sh doit.sh
python ./configure --dest-cpu=ppc64
exec(): 0509-036 Cannot load program /home/iojs/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1plus because of the following errors:
        0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
        0509-152   Member libmpc.so.3 is not found in archive 
Traceback (most recent call last):
  File "./configure", line 1472, in <module>
    check_compiler(output)
  File "./configure", line 716, in check_compiler
    ok, is_clang, clang_version, gcc_version = try_check_compiler(CXX, 'c++')
  File "./configure", line 615, in try_check_compiler
    proc.stdin.write('__clang__ __GNUC__ __GNUC_MINOR__ __GNUC_PATCHLEVEL__ '
IOError: [Errno 32] Broken pipe
gmake: *** [build-ci] Error 1
mhdawson commented 6 years ago

Ok, ignore the previous comment. I see that I have at least one of the paths wrong trying again.

mhdawson commented 6 years ago

Or maybe I had things right after all as @refack just said the same thing.

mhdawson commented 6 years ago

Tried taking the gmake from one of our internal machines that might have worked with @gibfahn tests. And tried it on the CI machine, still fails with the same problem :(

The message was only slightly different:


bash-4.3$ export LIBPATH=/home/iojs/gcc-6.3.0-1/opt/freeware/lib
-bash-4.3$ gmake
exec(): 0509-036 Cannot load program gmake because of the following errors:
        0509-022 Cannot load module /opt/freeware/lib/libintl.a(libintl.so.8).
        0509-150   Dependent module /home/iojs/gcc-6.3.0-1/opt/freeware/lib/libiconv.a(shr4_64.o) could not be loaded.
        0509-152   Member shr4_64.o is not found in archive 
        0509-022 Cannot load module gmake.
        0509-150   Dependent module /opt/freeware/lib/libintl.a(libintl.so.8) could not be loaded.
        0509-022 Cannot load module .
-bash-4.3$ 
```1
with the difference being libintl.so.8 instead of libintl.so.1
gibfahn commented 6 years ago

Tried taking the gmake from one of our internal machines that might have worked with @gibfahn tests. And tried it on the CI machine, still fails with the same problem :(

Assuming you're talking about dal-pax, then it worked fine (feel free to test on that machine). Given that they're both AIX 6 boxes with Bull Freeware rpms and gcc 4.8 (I didn't update any of the packages on that machine) it should be pretty simple to work out what the difference between them is.

mhdawson commented 6 years ago

I finally found something that works. Still want to get some feedback from the internal AIX team to see if it makes sense.

The environment is set as follows:

 export LIBPATH=/home/iojs/gmake/opt/freeware/lib:/home/iojs/gcc-6.3.0-1/opt/freeware/lib
 export PATH="/home/iojs/gcc-6.3.0-1/opt/freeware/bin:$PATH"
 export CC="ccache `which gcc`" CXX="ccache `which g++`" CXX_host="ccache `which g++`"

with the difference from before being the addition of /home/iojs/gmake/opt/freeware/lib where I've copied the libiconv.a from the local system. The net is that we use the libiconv.a that gmake needs as opposed to the one that comes from the bundle build for gcc.

Have tested from command line, next is to see if it works from the CI.

mhdawson commented 6 years ago

Running here: https://ci.nodejs.org/job/node-test-commit-aix-mdawson/nodes=aix61-ppc64-x/10/

gibfahn commented 6 years ago

The net is that we use the libiconv.a that gmake needs as opposed to the one that comes from the bundle build for gcc.

Do you know why the gmake on the test machines is different to the one on dal-pax?

mhdawson commented 6 years ago

No idea and in fact even copying over the gmake to the community machine did not help as I tried that first. It is different though based on the output from dump -H

BethGriggs commented 6 years ago

@gibfahn, dal-pax is 6100-07, IIRC the community CI AIX machines (https://ci.nodejs.org/label/aix61-ppc64/) are both 6100-09. Maybe that is why we're seeing differences?

mhdawson commented 6 years ago

The upgrade has now been rolled out to test and release machines. Things seem to be building ok. Have a suggestion from AIX team on how to improve the config, will do that but likely after 10.x goes out.

mhdawson commented 6 years ago

We ran into one more problem. Turns out there were a few non-relative symbolic links and the process documented above resulted in those being missing in the deployed compilers.

We've talked to the AIX teams about fixing them so they are relative and have fixed up the links on the community machines with:

cp opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/libatomic.a opt/freeware/li
b/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/ppc64/libatomic.a
cp opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/libgcc_s.a opt/freeware/lib
/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/ppc64/libgcc_s.a
cp opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/libstdc++.a opt/freeware/li
b/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/ppc64/libstdc++.a
cp opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/libsupc++.a opt/freeware/li
b/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/ppc64/libsupc++.a

cp opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/libatomic.a opt/fre
eware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64/libatomic.a
cp opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/libgcc_s.a opt/free
ware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64/libgcc_s.a
cp opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/libstdc++.a opt/fre
eware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64/libstdc++.a
cp opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/libsupc++.a opt/fre
eware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64/libsupc++.a

cp opt/freeware/lib/libmpc.a opt/freeware/lib64/libmpc.a
cp opt/freeware/lib/libxml2.a opt/freeware/lib64/libxml2.a
cp opt/freeware/lib/libz.a opt/freeware/lib64/libz.a

The updated fixed binary is in the downloads directory on the ci machine as: V2-gcc-6.3.0-1.tar.gz

The next step is to update the manual install steps for AIX.

Trott commented 5 years ago

This is almost a year inactive at this point. Is it still a thing or has this been addressed and can be closed? @mhdawson @nodejs/platform-aix

sam-github commented 5 years ago

I think its still a thing. I've notes somewhere, I think we are still working on it, slowly. I'll check with @mhdawson again when he's back from vacation.

mhdawson commented 5 years ago

The remaining stuff I think is to update the manual install steps for AIX and then it can be closed.

AshCripps commented 4 years ago

@mhdawson is this the updated install steps you mentioned - https://github.com/nodejs/help/wiki/Installation#how-to-install-nodejs-via-binary-archive-on-aix.

If so, can this issue be closed?

mhdawson commented 4 years ago

I'm ok with closing.