Ansible request for AIX: Correct libc++ runtime environment.

aixtools commented 2 years ago

Historically, the builds requiring xlc16 (e.g., openj9) have also added two OSS packages: libgcc and libgcc8.
These were needed because the (beta/GA) release of xlc16 - did not have the rte library package libc++.rte.
This library is provided by IBM, both in the xlC16 package, as in a separate 'runt-time-only' package.
As, such, NOT using the IBM provided library (in particular libatomic.a) may introduce unexpected issues.
[x] Missing install
[x] Bug in ansible playbook
[ ] Request for new playbook addition

Details:

Current:

p159a01:/ # rpm -qa | grep gcc
libgcc-8-1.ppc
libgcc8-8.3.0-4.ppc
p159a01:/ # rpm -ql libgcc
/opt/freeware/lib/libatomic.a
/opt/freeware/lib/libgcc_s.a
/opt/freeware/lib/pthread/libatomic.a
/opt/freeware/lib/pthread/libgcc_s.a
/opt/freeware/lib/pthread/ppc64/libatomic.a
/opt/freeware/lib/pthread/ppc64/libgcc_s.a
/opt/freeware/lib64/libatomic.a
/opt/freeware/lib64/libgcc_s.a
p159a01:/ # rpm -ql libgcc8
/opt/freeware/doc/libgcc8-8.3.0
/opt/freeware/doc/libgcc8-8.3.0/COPYING
/opt/freeware/doc/libgcc8-8.3.0/COPYING.LIB
/opt/freeware/doc/libgcc8-8.3.0/COPYING3
/opt/freeware/doc/libgcc8-8.3.0/COPYING3.LIB
/opt/freeware/lib/gcc
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8/libatomic.a
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8/libgcc_s.a
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8/ppc64
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8/ppc64/libatomic.a
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8/ppc64/libgcc_s.a
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8/pthread
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8/pthread/libatomic.a
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8/pthread/libgcc_s.a
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8/pthread/ppc64
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8/pthread/ppc64/libatomic.a
/opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8/pthread/ppc64/libgcc_s.a

Desired:

jenkins@adopt08:[/home/jenkins/bin]lslpp -f libc++.rte
  Fileset               File
  ----------------------------------------------------------------------------
Path: /usr/lib/objrepos
  libc++.rte 16.1.0.2   /usr/lpp/xlC/lib/aix61/libatomic.a
                        /usr/lpp/xlC
                        /usr/lib/libatomic.a -> /usr/lpp/xlC/lib/libatomic.a
                        /usr/lib/libc++.a -> /usr/lpp/xlC/lib/libc++.a
                        /usr/lpp/xlC/lib
                        /usr/lpp/xlC/lib/aix61/libc++.a
                        /usr/lpp/xlC/lib/aix61

sxa commented 2 years ago

What is the proposed change here? Install the libc++ runtime (I'm surprised it works at all if that's not installed - I wouldn't expect the GNU one to work in place of it)

aixtools commented 2 years ago

The proposal is to remove the AIXToolbox one and verify libc++ from the XLC16 package is updated, i.e., file
my concern comes from the recommendation in slack threads (that I assume are based on Adoptium documentation (a link was mentioned in the slack convo, AND, because I do not know where to look to see what parameters jenkins builds provides) include this: export LIBPATH=/opt/freeware/lib/pthread/ppc64:/opt/freeware/lib:/usr/lib
Agree: confusing (fileset name, and the file being targetted - libatomic.a

Wrong fileset:

p159a01:/ # lslpp -f libc++.rte
Fileset               File
----------------------------------------------------------------------------
Path: /usr/lib/objrepos
libc++.rte 16.1.0.0   /usr/lpp/xlC
                    /usr/lib/libc++.a -> /usr/lpp/xlC/lib/libc++.a
                    /usr/lpp/xlC/lib
                    /usr/lpp/xlC/lib/aix61/libc++.a
                    /usr/lpp/xlC/lib/aix61

Updated fileset:

jenkins@p8-aix1-adopt03:[/home/jenkins/bin]lslpp -f libc++.rte
Fileset               File
----------------------------------------------------------------------------
Path: /usr/lib/objrepos
libc++.rte 16.1.0.3   /usr/lpp/xlC/lib/aix61/libatomic.a
                    /usr/lpp/xlC
                    /usr/lib/libatomic.a -> /usr/lpp/xlC/lib/libatomic.a
                    /usr/lib/libc++.a -> /usr/lpp/xlC/lib/libc++.a
                    /usr/lpp/xlC/lib
                    /usr/lpp/xlC/lib/aix61/libc++.a
                    /usr/lpp/xlC/lib/aix61

Basically, since this is missing from the playbook, and xlC16 is not updated regularly - we run into this: (excluding IBM hosted systems (but both at 16.1.0.0, fyi):

==============
root@nim.bak:[/home/root]dsh-adopt.ksh lslpp -Lcq libc++.rte
adopt01:
libc++.rte:libc++.rte:16.1.0.0: : :C: :IBM XL C++ Runtime for AIX 7.1 and later: : : : : : :0:0:/:
==============
adopt02:
libc++.rte:libc++.rte:16.1.0.0: : :C: :IBM XL C++ Runtime for AIX 7.1 and later: : : : : : :0:0:/:
==============
adopt03:
libc++.rte:libc++.rte:16.1.0.3: : :C: :IBM XL C++ Runtime for AIX 7.1 and later : : : : : : :1:0:/:
==============
adopt04:
libc++.rte:libc++.rte:16.1.0.0: : :C: :IBM XL C++ Runtime for AIX 7.1 and later: : : : : : :0:0:/:
==============
adopt05:
libc++.rte:libc++.rte:16.1.0.0: : :C: :IBM XL C++ Runtime for AIX 7.1 and later: : : : : : :0:0:/:
==============
adopt06:
libc++.rte:libc++.rte:16.1.0.6: : :C:F:IBM XL C++ Runtime for AIX 7.1 and later: : : : : : :0:0:/:
==============
adopt07:
libc++.rte:libc++.rte:16.1.0.0: : :C: :IBM XL C++ Runtime for AIX 7.1 and later: : : : : : :0:0:/:
==============
adopt08:
libc++.rte:libc++.rte:16.1.0.7: : :C:F:IBM XL C++ Runtime for AIX 7.1 and later: : : : : : :0:0:/:
==============

Summary:

(Best guess): Historically, there was a problem - xlc16 (libc++.rte) did not provide the libatomic.a library - and so one was found in an AIXToolbox GCC libc++ package.
I propose to remove the RPM packages, and their installation steps in the playbooks.

sxa commented 2 years ago

I'm not quite clear on what the output from dsh-adopt.ksh lslpp -Lcq libc++.rte is showing me but we do not directly install libstdc++ from the playbooks as far as I'm aware but it is pulled in due to being required by a number of other packages, including cmake which is required for building OpenJ9 and python2 which will be required by yum so it's unlikely to be trivial to remove without causing side effects:

jenkins@p8-aix1-adopt01:[/home/jenkins]rpm -q --whatrequires libstdc++
ncurses-6.2-2.ppc
gmp-6.1.2-1.ppc
gettext-0.20.2-1.ppc
python-2.7.18-1.ppc
pcre-8.44-1.ppc
libtiff-4.1.0-1.ppc
libstdc++-devel-8-1.ppc
cmake-3.16.0-2.ppc
jenkins@p8-aix1-adopt01:[/home/jenkins]

Perhaps as an alternative we should experiment and try running the test suites without freeware in the PATH or LIBPATH at all.

aixtools commented 2 years ago

At issue is creating a dependency for the java's being built on an OSS package - that has been introduced because it was forgotten during the install of xlCv16.
There is a RTE (e.g., xlc.rte.16.1.0.7-IBM-xlC-AIX-FP007.tar.Z) package that is available for download for applications using xlCv16 (that, depending on the application, might be needed to fulfill RTE requirements.
my concern is not that it is needed. my concern is that it is unknown whether java needs it (or not), and if needed for java - where is the documentation that OSS libstdc++ is needed.

adoptium / infrastructure

Ansible request for AIX: Correct libc++ runtime environment. #2421