Compilation with petsc 3.14 on archer2

[stevendargaville]

In testing out a new petsc 3.14 build on archer2, fluidity seems to be failing to link with the error:

Error was: /usr/bin/ld: ./lib/libfluidity.a(Solvers.o): in function __solvers_MOD_setup_pc_from_options': Solvers.F90:(.text+0x7ebd): undefined reference topcasmgetsubksp_'

Commenting out the call to PCASMGetSubKSP seems to get it to build successfully. I'm just guessing that the interface to that petsc call may have changed in 3.14; previous compilations with 3.13 were successful.

[stephankramer]

Hm, odd: I do have a succesfull build with 3.14. Also the routine still exists in current 3.15 : https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/PC/PCASMGetSubKSP.html If I look in the petsc library for the symbols:

$ nm -D /usr/lib/petscdir/3.14/lib/libpetsc_real.so | grep -i pcasmgetsubksp
0000000000bff5c0 T PCASMGetSubKSP
0000000000c06430 T pcasmgetsubksp1_
0000000000c066b0 T pcasmgetsubksp2_
0000000000c066c0 T pcasmgetsubksp3_
0000000000c066d0 T pcasmgetsubksp4_
0000000000c066e0 T pcasmgetsubksp5_
0000000000c066f0 T pcasmgetsubksp6_
0000000000c06700 T pcasmgetsubksp7_
0000000000c06710 T pcasmgetsubksp8_

and

$ nm -A femtools/Solvers.o | grep -i pcasm
femtools/Solvers.o:                 U pcasmgetsubksp1_

So there seems to be some "advanced" name mangling going on. Maybe you are having a mismatch between the compiler used to build petsc on archer, and the one you're using to build fluidity?

[stevendargaville]

So there seems to be some "advanced" name mangling going on. Maybe you are having a mismatch between the compiler used to build petsc on archer, and the one you're using to build fluidity?

Hmm ok, looks like this is an archer specific issue then. @tmbgreaves, would this be worth raising with the archer team to check which compiler they used to build petsc 3.14? Having a peak into the symbols of the 3.14 on archer2, it does seem to find some things that look similar to Stephan above, which seems to suggest the PC routine is there, but I am by no means an expert at interpreting!

$ objdump -t /work/z19/shared/petsc-hypre/libs/petsc/3.14.2/CRAYCLANG/10.0/lib/libpetsc.a | grep -i pcasmgetsubksp

000000000000001cc l     O .rodata.str1.1    000000000000000f .L__func__.PCASMGetSubKSP
0000000000000c33 l     O .rodata.str1.1 0000000000000013 .L__func__.PCASMGetSubKSP_ASM
00000000000068b0 l     F .text  00000000000000d8 PCASMGetSubKSP_ASM
0000000000000480 g     F .text  00000000000000bc PCASMGetSubKSP
0000000000000000         *UND*  0000000000000000 PCASMGetSubKSP
00000000000002b0 g     F .text  00000000000003d5 pcasmgetsubksp1_
0000000000000690 g     F .text  0000000000000005 pcasmgetsubksp2_

[tmbgreaves]

Thanks @stevendargaville , I'll check with them and report back here.

[tmbgreaves]

Our Archer contact reports:

I just used the defaults, i.e. not the new CPE/21.03 environment.

The Cray installation scripts build the libraries using both Cray and GCC compilers - the way that modules are built is supposed to automagically select the right one based on whether a user currently has the Cray or GNU programming environment loaded ...

Grepping the installation logs shows:

 Version: Cray clang version 10.0.4

(ffb772459f6195ccd74395703e557b253fc8ee27) Version: Cray Fortran : Version 10.0.4 Fortran compiler version: Cray Fortran : Version 10.0.4

and

 Version: gcc (GCC) 9.3.0 20200312 (Cray Inc.)
 Version: g++ (GCC) 9.3.0 20200312 (Cray Inc.)
 Version: GNU Fortran (GCC) 9.3.0 20200312 (Cray Inc.)

[stephankramer]

Closing due to inactivity