search

Electrostatics / FETK

Fork of the Finite Element ToolKit from fetk.org.
Other
4 stars 4 forks source link

ld linker error for cgcode on newest xcode/llvm version 13.0.0 #63

Open BJWiley233 opened 2 years ago

BJWiley233 commented 2 years ago

We can build static libraries but when building and linking shared libraries we get the "canonical" ld: symbol(s) not found for architecture x86_64 errors.

example invocation of compiling a cgcode src:

"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang" -cc1 -triple x86_64-apple-macosx11.0.0 -Wundef-prefix=TARGET_OS_ -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -Werror=implicit-function-declaration -emit-obj -mrelax-all --mrelax-relocations -disable-free -disable-llvm-verifier -discard-value-names -main-file-name depsln.c -mrelocation-model pic -pic-level 2 -mframe-pointer=all -fno-strict-return -fno-rounding-math -munwind-tables -target-sdk-version=12.1 -fvisibility-inlines-hidden-static-local-var -target-cpu penryn -tune-cpu generic -debugger-tuning=lldb -target-linker-version 711 -v -resource-dir /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0 -dependency-file CMakeFiles/cgcode.dir/depsln.c.o.d -skip-unused-modulemap-deps -MT punc/src/cgcode/CMakeFiles/cgcode.dir/depsln.c.o -sys-header-deps -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -D cgcode_EXPORTS -I /Users/brian/tools/FETK/punc/../maloc/src/base -I /Users/brian/tools/FETK/punc/../maloc/src/psh -I /Users/brian/tools/FETK/punc/../maloc/src/vsh -I /Users/brian/tools/FETK/punc/../maloc/src/vsys -I /Users/brian/tools/FETK/punc/src/vf2c -I /Users/brian/tools/FETK/punc/src/vf2c/punc -I /Users/brian/tools/FETK/punc/src/cgcode -I /usr/local/Cellar/libomp/13.0.0/include -I /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include -internal-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/local/include -internal-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include -internal-externc-isystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include -Wno-reorder-init-list -Wno-implicit-int-float-conversion -Wno-c99-designator -Wno-final-dtor-non-final-class -Wno-extra-semi-stmt -Wno-misleading-indentation -Wno-quoted-include-in-framework-header -Wno-implicit-fallthrough -Wno-enum-enum-conversion -Wno-enum-float-conversion -Wno-elaborated-enum-base -fdebug-compilation-dir /Users/brian/tools/FETK/build/punc/src/cgcode -ferror-limit 19 -stack-protector 1 -fstack-check -mdarwin-stkchk-strong-link -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -fcommon -fcolor-diagnostics -clang-vendor-feature=+nullptrToBoolConversion -clang-vendor-feature=+messageToSelfInClassMethodIdReturnType -clang-vendor-feature=+disableInferNewAvailabilityFromInit -clang-vendor-feature=+disableNeonImmediateRangeCheck -clang-vendor-feature=+disableNonDependentMemberExprInCurrentInstantiation -fno-odr-hash-protocols -clang-vendor-feature=+revert09abecef7bbf -mllvm -disable-aligned-alloc-awareness=1 -mllvm -enable-dse-memoryssa=0 -o CMakeFiles/cgcode.dir/depsln.c.o -x c /Users/brian/tools/FETK/punc/src/cgcode/depsln.c

And invocation of linker for dylib:

"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -dylib -arch x86_64 -dylib_install_name @rpath/libcgcode.dylib -platform_version macos 11.0.0 12.1 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -o libcgcode.dylib -L/usr/local/Cellar/libomp/13.0.0/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/lib -L/usr/local/opt/tcl-tk/lib -L/usr/local/opt/tcl-tk/lib -L. -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/lib -lomp -headerpad_max_install_names CMakeFiles/cgcode.dir/scgdrv.c.o CMakeFiles/cgcode.dir/scg.c.o CMakeFiles/cgcode.dir/scr.c.o CMakeFiles/cgcode.dir/scrind.c.o CMakeFiles/cgcode.dir/spcg.c.o CMakeFiles/cgcode.dir/scgnr.c.o CMakeFiles/cgcode.dir/scgne.c.o CMakeFiles/cgcode.dir/spcgnr.c.o CMakeFiles/cgcode.dir/spcgne.c.o CMakeFiles/cgcode.dir/sppcg.c.o CMakeFiles/cgcode.dir/spcgca.c.o CMakeFiles/cgcode.dir/sdpchb.c.o CMakeFiles/cgcode.dir/scbfix.c.o CMakeFiles/cgcode.dir/sckchb.c.o CMakeFiles/cgcode.dir/scgchk.c.o CMakeFiles/cgcode.dir/sonest.c.o CMakeFiles/cgcode.dir/msstop.c.o CMakeFiles/cgcode.dir/r1mach.c.o CMakeFiles/cgcode.dir/sratqr.c.o CMakeFiles/cgcode.dir/dcgdrv.c.o CMakeFiles/cgcode.dir/dcg.c.o CMakeFiles/cgcode.dir/dcr.c.o CMakeFiles/cgcode.dir/dcrind.c.o CMakeFiles/cgcode.dir/dpcg.c.o CMakeFiles/cgcode.dir/dcgnr.c.o CMakeFiles/cgcode.dir/dcgne.c.o CMakeFiles/cgcode.dir/dpcgnr.c.o CMakeFiles/cgcode.dir/dpcgne.c.o CMakeFiles/cgcode.dir/dppcg.c.o CMakeFiles/cgcode.dir/dpcgca.c.o CMakeFiles/cgcode.dir/ddpchb.c.o CMakeFiles/cgcode.dir/dcbfix.c.o CMakeFiles/cgcode.dir/dckchb.c.o CMakeFiles/cgcode.dir/dcgchk.c.o CMakeFiles/cgcode.dir/donest.c.o CMakeFiles/cgcode.dir/mdstop.c.o CMakeFiles/cgcode.dir/d1mach.c.o CMakeFiles/cgcode.dir/dratqr.c.o CMakeFiles/cgcode.dir/depsln.c.o -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/lib/darwin/libclang_rt.osx.a

Gives these errors:

Undefined symbols for architecture x86_64:
  "_daxpy_", referenced from:
      _dcg_ in dcg.c.o
      _dcr_ in dcr.c.o
      _dpcg_ in dpcg.c.o
      _dcgnr_ in dcgnr.c.o
      _dcgne_ in dcgne.c.o
      _dpcgnr_ in dpcgnr.c.o
      _dpcgne_ in dpcgne.c.o
      ...
  "_dcopy_", referenced from:
      _dcg_ in dcg.c.o
      _dcr_ in dcr.c.o
      _dpcgca_ in dpcgca.c.o
      _dcbfix_ in dcbfix.c.o
  "_ddot_", referenced from:
      _dcg_ in dcg.c.o
      _dcr_ in dcr.c.o
      _dcrind_ in dcrind.c.o
      _dpcg_ in dpcg.c.o
      _dppcg_ in dppcg.c.o
      _dpcgca_ in dpcgca.c.o
  "_dnrm2_", referenced from:
      _dcg_ in dcg.c.o
      _dcr_ in dcr.c.o
      _dcrind_ in dcrind.c.o
      _dpcg_ in dpcg.c.o
      _dcgnr_ in dcgnr.c.o
      _dcgne_ in dcgne.c.o
      _dpcgnr_ in dpcgnr.c.o
      ...
  "_do_fio", referenced from:
      _scgdrv_ in scgdrv.c.o
      _scg_ in scg.c.o
      _scr_ in scr.c.o
      _scrind_ in scrind.c.o
      _spcg_ in spcg.c.o
      _scgnr_ in scgnr.c.o
      _scgne_ in scgne.c.o
      ...
  "_e_wsfe", referenced from:
      _scgdrv_ in scgdrv.c.o
      _scg_ in scg.c.o
      _scr_ in scr.c.o
      _scrind_ in scrind.c.o
      _spcg_ in spcg.c.o
      _scgnr_ in scgnr.c.o
      _scgne_ in scgne.c.o
      ...
  "_pow_dd", referenced from:
      _sdpchb_ in sdpchb.c.o
      _ddpchb_ in ddpchb.c.o
  "_pow_di", referenced from:
      _ddpchb_ in ddpchb.c.o
  "_pow_ri", referenced from:
      _sdpchb_ in sdpchb.c.o
  "_s_stop", referenced from:
      _sckchb_ in sckchb.c.o
      _scgchk_ in scgchk.c.o
      _dckchb_ in dckchb.c.o
      _dcgchk_ in dcgchk.c.o
  "_s_wsfe", referenced from:
      _scgdrv_ in scgdrv.c.o
      _scg_ in scg.c.o
      _scr_ in scr.c.o
      _scrind_ in scrind.c.o
      _spcg_ in spcg.c.o
      _scgnr_ in scgnr.c.o
      _scgne_ in scgne.c.o
      ...
  "_saxpy_", referenced from:
      _scg_ in scg.c.o
      _scr_ in scr.c.o
      _spcg_ in spcg.c.o
      _scgnr_ in scgnr.c.o
      _scgne_ in scgne.c.o
      _spcgnr_ in spcgnr.c.o
      _spcgne_ in spcgne.c.o
      ...
  "_scopy_", referenced from:
      _scg_ in scg.c.o
      _scr_ in scr.c.o
      _spcgca_ in spcgca.c.o
      _scbfix_ in scbfix.c.o
  "_sdot_", referenced from:
      _scg_ in scg.c.o
      _scr_ in scr.c.o
      _scrind_ in scrind.c.o
      _spcg_ in spcg.c.o
      _sppcg_ in sppcg.c.o
      _spcgca_ in spcgca.c.o
  "_snrm2_", referenced from:
      _scg_ in scg.c.o
      _scr_ in scr.c.o
      _scrind_ in scrind.c.o
      _spcg_ in spcg.c.o
      _scgnr_ in scgnr.c.o
      _scgne_ in scgne.c.o
      _spcgnr_ in spcgnr.c.o
      ...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [punc/src/cgcode/libcgcode.dylib] Error 1
make[1]: *** [punc/src/cgcode/CMakeFiles/cgcode.dir/all] Error 2
BJWiley233 commented 2 years ago

There is no problem with vf2c:

Linking C shared library libvf2c.dylib
"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -lto_library /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -dynamic -dylib -arch x86_64 -dylib_install_name @rpath/libvf2c.dylib -platform_version macos 11.6.0 11.3 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk -o libvf2c.dylib -L/usr/local/Cellar/libomp/13.0.0/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/lib -L. -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/lib -lomp -headerpad_max_install_names CMakeFiles/vf2c.dir/f77vers.c.o CMakeFiles/vf2c.dir/i77vers.c.o CMakeFiles/vf2c.dir/s_rnge.c.o CMakeFiles/vf2c.dir/abort_.c.o CMakeFiles/vf2c.dir/exit_.c.o CMakeFiles/vf2c.dir/getarg_.c.o CMakeFiles/vf2c.dir/iargc_.c.o CMakeFiles/vf2c.dir/getenv_.c.o CMakeFiles/vf2c.dir/signal_.c.o CMakeFiles/vf2c.dir/s_stop.c.o CMakeFiles/vf2c.dir/s_paus.c.o CMakeFiles/vf2c.dir/system_.c.o CMakeFiles/vf2c.dir/cabs.c.o CMakeFiles/vf2c.dir/derf_.c.o CMakeFiles/vf2c.dir/derfc_.c.o CMakeFiles/vf2c.dir/erf_.c.o CMakeFiles/vf2c.dir/erfc_.c.o CMakeFiles/vf2c.dir/sig_die.c.o CMakeFiles/vf2c.dir/uninit.c.o CMakeFiles/vf2c.dir/pow_ci.c.o CMakeFiles/vf2c.dir/pow_dd.c.o CMakeFiles/vf2c.dir/pow_di.c.o CMakeFiles/vf2c.dir/pow_hh.c.o CMakeFiles/vf2c.dir/pow_ii.c.o CMakeFiles/vf2c.dir/pow_ri.c.o CMakeFiles/vf2c.dir/pow_zi.c.o CMakeFiles/vf2c.dir/pow_zz.c.o CMakeFiles/vf2c.dir/c_abs.c.o CMakeFiles/vf2c.dir/c_cos.c.o CMakeFiles/vf2c.dir/c_div.c.o CMakeFiles/vf2c.dir/c_exp.c.o CMakeFiles/vf2c.dir/c_log.c.o CMakeFiles/vf2c.dir/c_sin.c.o CMakeFiles/vf2c.dir/c_sqrt.c.o CMakeFiles/vf2c.dir/z_abs.c.o CMakeFiles/vf2c.dir/z_cos.c.o CMakeFiles/vf2c.dir/z_div.c.o CMakeFiles/vf2c.dir/z_exp.c.o CMakeFiles/vf2c.dir/z_log.c.o CMakeFiles/vf2c.dir/z_sin.c.o CMakeFiles/vf2c.dir/z_sqrt.c.o CMakeFiles/vf2c.dir/r_abs.c.o CMakeFiles/vf2c.dir/r_acos.c.o CMakeFiles/vf2c.dir/r_asin.c.o CMakeFiles/vf2c.dir/r_atan.c.o CMakeFiles/vf2c.dir/r_atn2.c.o CMakeFiles/vf2c.dir/r_cnjg.c.o CMakeFiles/vf2c.dir/r_cos.c.o CMakeFiles/vf2c.dir/r_cosh.c.o CMakeFiles/vf2c.dir/r_dim.c.o CMakeFiles/vf2c.dir/r_exp.c.o CMakeFiles/vf2c.dir/r_imag.c.o CMakeFiles/vf2c.dir/r_int.c.o CMakeFiles/vf2c.dir/r_lg10.c.o CMakeFiles/vf2c.dir/r_log.c.o CMakeFiles/vf2c.dir/r_mod.c.o CMakeFiles/vf2c.dir/r_nint.c.o CMakeFiles/vf2c.dir/r_sign.c.o CMakeFiles/vf2c.dir/r_sin.c.o CMakeFiles/vf2c.dir/r_sinh.c.o CMakeFiles/vf2c.dir/r_sqrt.c.o CMakeFiles/vf2c.dir/r_tan.c.o CMakeFiles/vf2c.dir/r_tanh.c.o CMakeFiles/vf2c.dir/d_abs.c.o CMakeFiles/vf2c.dir/d_acos.c.o CMakeFiles/vf2c.dir/d_asin.c.o CMakeFiles/vf2c.dir/d_atan.c.o CMakeFiles/vf2c.dir/d_atn2.c.o CMakeFiles/vf2c.dir/d_cnjg.c.o CMakeFiles/vf2c.dir/d_cos.c.o CMakeFiles/vf2c.dir/d_cosh.c.o CMakeFiles/vf2c.dir/d_dim.c.o CMakeFiles/vf2c.dir/d_exp.c.o CMakeFiles/vf2c.dir/d_imag.c.o CMakeFiles/vf2c.dir/d_int.c.o CMakeFiles/vf2c.dir/d_lg10.c.o CMakeFiles/vf2c.dir/d_log.c.o CMakeFiles/vf2c.dir/d_mod.c.o CMakeFiles/vf2c.dir/d_nint.c.o CMakeFiles/vf2c.dir/d_prod.c.o CMakeFiles/vf2c.dir/d_sign.c.o CMakeFiles/vf2c.dir/d_sin.c.o CMakeFiles/vf2c.dir/d_sinh.c.o CMakeFiles/vf2c.dir/d_sqrt.c.o CMakeFiles/vf2c.dir/d_tan.c.o CMakeFiles/vf2c.dir/d_tanh.c.o CMakeFiles/vf2c.dir/i_abs.c.o CMakeFiles/vf2c.dir/i_dim.c.o CMakeFiles/vf2c.dir/i_dnnt.c.o CMakeFiles/vf2c.dir/i_indx.c.o CMakeFiles/vf2c.dir/i_len.c.o CMakeFiles/vf2c.dir/i_mod.c.o CMakeFiles/vf2c.dir/i_nint.c.o CMakeFiles/vf2c.dir/i_sign.c.o CMakeFiles/vf2c.dir/lbitbits.c.o CMakeFiles/vf2c.dir/lbitshft.c.o CMakeFiles/vf2c.dir/h_abs.c.o CMakeFiles/vf2c.dir/h_dim.c.o CMakeFiles/vf2c.dir/h_dnnt.c.o CMakeFiles/vf2c.dir/h_indx.c.o CMakeFiles/vf2c.dir/h_len.c.o CMakeFiles/vf2c.dir/h_mod.c.o CMakeFiles/vf2c.dir/h_nint.c.o CMakeFiles/vf2c.dir/h_sign.c.o CMakeFiles/vf2c.dir/l_ge.c.o CMakeFiles/vf2c.dir/l_gt.c.o CMakeFiles/vf2c.dir/l_le.c.o CMakeFiles/vf2c.dir/l_lt.c.o CMakeFiles/vf2c.dir/hl_ge.c.o CMakeFiles/vf2c.dir/hl_gt.c.o CMakeFiles/vf2c.dir/hl_le.c.o CMakeFiles/vf2c.dir/hl_lt.c.o CMakeFiles/vf2c.dir/ef1asc_.c.o CMakeFiles/vf2c.dir/ef1cmc_.c.o CMakeFiles/vf2c.dir/f77_aloc.c.o CMakeFiles/vf2c.dir/s_cat.c.o CMakeFiles/vf2c.dir/s_cmp.c.o CMakeFiles/vf2c.dir/s_copy.c.o CMakeFiles/vf2c.dir/dtime_.c.o CMakeFiles/vf2c.dir/etime_.c.o CMakeFiles/vf2c.dir/backspac.c.o CMakeFiles/vf2c.dir/close.c.o CMakeFiles/vf2c.dir/dfe.c.o CMakeFiles/vf2c.dir/dolio.c.o CMakeFiles/vf2c.dir/due.c.o CMakeFiles/vf2c.dir/endfile.c.o CMakeFiles/vf2c.dir/err.c.o CMakeFiles/vf2c.dir/fmt.c.o CMakeFiles/vf2c.dir/fmtlib.c.o CMakeFiles/vf2c.dir/ftell_.c.o CMakeFiles/vf2c.dir/iio.c.o CMakeFiles/vf2c.dir/ilnw.c.o CMakeFiles/vf2c.dir/inquire.c.o CMakeFiles/vf2c.dir/lread.c.o CMakeFiles/vf2c.dir/lwrite.c.o CMakeFiles/vf2c.dir/open.c.o CMakeFiles/vf2c.dir/rdfmt.c.o CMakeFiles/vf2c.dir/rewind.c.o CMakeFiles/vf2c.dir/rsfe.c.o CMakeFiles/vf2c.dir/rsli.c.o CMakeFiles/vf2c.dir/rsne.c.o CMakeFiles/vf2c.dir/sfe.c.o CMakeFiles/vf2c.dir/sue.c.o CMakeFiles/vf2c.dir/typesize.c.o CMakeFiles/vf2c.dir/uio.c.o CMakeFiles/vf2c.dir/util.c.o CMakeFiles/vf2c.dir/wref.c.o CMakeFiles/vf2c.dir/wrtfmt.c.o CMakeFiles/vf2c.dir/wsfe.c.o CMakeFiles/vf2c.dir/wsle.c.o CMakeFiles/vf2c.dir/wsne.c.o CMakeFiles/vf2c.dir/xwsne.c.o CMakeFiles/vf2c.dir/main.c.o -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/13.0.0/lib/darwin/libclang_rt.osx.a
[ 28%] Built target vf2c
BJWiley233 commented 2 years ago

ok same thing for pmg library. ld really does not like code translated from fortran to c with f2c.

BJWiley233 commented 2 years ago

Yea it's an Apple issue. I don't think they are really shipping new Xcode or their SDK with accelerate vecLib framework any longer so the linker couldn't see the libblas routines. Had to link to openblas. I think they are seeing these issue in R as well, albeit the responder sounds pretty rude. Another issue I did see was when linking cgcode it could not find the f2c library routines. So functions like pow_dd could not be found. I had to also link with -lf2c to get linker to work so not really sure if we should be "sedding" out #include "f2c.h" when running f2c?

BJWiley233 commented 2 years ago

Now there is another issue in that there are some functions that are completely undefined in the pmg fortran code. For instance pmg can call c version of linesearch_ routine because linesearch is actually defined in the fortran code.

However ngsd.c amongst other srcs, tries to call c_scal__ and dc_scal__ but these functions defined nowhere in this repo. They are in the main vpmg.h header but that doesn't do any good if they are not defined. I think we are missing some of Michael Holst's fortran sources.

BJWiley233 commented 2 years ago

I did a quick check of the routines that exist from downloading http://www.fetk.org/codes/download/punc-1.5.tar.gz and grepping for their subroutines from the vpmg.h. Some are part of blas such as ddot and idamax so that is fine but you can see we are missing a few (the ones that have just single column). Some of these are in the fortran code in the test punc/tools/tests/pmg folder such as packmg, delget, & fillco; some interesting ones is defined as function instead of subroutine so f2c gets these. But we are missing these at least these 4 and so the linker complains (it also complains about rand_ and tsecnd_ which are also defined in the test folder punc/tools/tests/pmg).

c_scal
dc_scal
c_vec
dc_vec

Here are the results I got from searching PUNC/pmg fortran code:

# grep the vpmg.h header at https://github.com/Electrostatics/FETK/blob/main/punc/src/vf2c/punc/vpmg.h
$ fortran_subs=($(grep VEXTERNC vpmg.h | cut -d"(" -f1 | awk '{gsub("[_]{1,2}$","",$3); print $3}'))

# after downloading PUNC v1.5
$ for sub in ${fortran_subs[@]}; do printf "${sub}\t"; grep -i "subroutine ${sub} (" /Users/brian/Downloads/punc/src/pmg/src_f77/*.f; grep -i "subroutine ${sub}(" /Users/brian/Downloads/punc/src/pmg/src_f77/*.f; grep -i "function ${sub}(" /Users/brian/Downloads/punc/src/pmg/src_f77/*.f; echo ; done

builda  /Users/brian/Downloads/punc/src/pmg/src_f77/buildAd.f:      subroutine buildA (nx,ny,nz,ipkey,mgdisc,numdia,

builda_fv       /Users/brian/Downloads/punc/src/pmg/src_f77/buildAd.f:      subroutine buildA_fv (nx,ny,nz,ipkey,numdia,

builda_fe       /Users/brian/Downloads/punc/src/pmg/src_f77/buildAd.f:      subroutine buildA_fe (nx,ny,nz,ipkey,numdia,

buildband       /Users/brian/Downloads/punc/src/pmg/src_f77/buildBd.f:      subroutine buildband(key,nx,ny,nz,ipc,rpc,ac,ipcB,rpcB,acB)

buildband1_7    /Users/brian/Downloads/punc/src/pmg/src_f77/buildBd.f:      subroutine buildband1_7 (nx,ny,nz,ipc,rpc,

buildband1_27   /Users/brian/Downloads/punc/src/pmg/src_f77/buildBd.f:      subroutine buildband1_27 (nx,ny,nz,ipc,rpc,

buildg  /Users/brian/Downloads/punc/src/pmg/src_f77/buildGd.f:      subroutine buildG (nxf,nyf,nzf,nxc,nyc,nzc,numdia,pcFF,acFF,ac)

buildg_1        /Users/brian/Downloads/punc/src/pmg/src_f77/buildGd.f:      subroutine buildG_1 (nxf,nyf,nzf,nx,ny,nz,

buildg_7        /Users/brian/Downloads/punc/src/pmg/src_f77/buildGd.f:      subroutine buildG_7 (nxf,nyf,nzf,nx,ny,nz,

buildg_27       /Users/brian/Downloads/punc/src/pmg/src_f77/buildGd.f:      subroutine buildG_27 (nxf,nyf,nzf,nx,ny,nz,

buildp  /Users/brian/Downloads/punc/src/pmg/src_f77/buildPd.f:      subroutine buildP (nxf,nyf,nzf,nxc,nyc,nzc,mgprol,

buildp_trilin   /Users/brian/Downloads/punc/src/pmg/src_f77/buildPd.f:      subroutine buildP_trilin (nxf,nyf,nzf,nxc,nyc,nzc,pc,xf,yf,zf)

buildpb_trilin  /Users/brian/Downloads/punc/src/pmg/src_f77/buildPd.f:      subroutine buildPb_trilin (nxf,nyf,nzf,nxc,nyc,nzc,

buildp_op7      /Users/brian/Downloads/punc/src/pmg/src_f77/buildPd.f:      subroutine buildP_op7 (nxf,nyf,nzf,nxc,nyc,nzc,ipc,rpc,ac,pc)

buildpb_op7     /Users/brian/Downloads/punc/src/pmg/src_f77/buildPd.f:      subroutine buildPb_op7 (nxf,nyf,nzf,nxc,nyc,nzc,

buildp_op27     /Users/brian/Downloads/punc/src/pmg/src_f77/buildPd.f:      subroutine buildP_op27 (nxf,nyf,nzf,nxc,nyc,nzc,ipc,rpc,ac,pc)

buildpb_op27    /Users/brian/Downloads/punc/src/pmg/src_f77/buildPd.f:      subroutine buildPb_op27 (nxf,nyf,nzf,nxc,nyc,nzc,

buildp_modop7   /Users/brian/Downloads/punc/src/pmg/src_f77/buildPd.f:      subroutine buildP_modop7 (nxf,nyf,nzf,nxc,nyc,nzc,ipc,rpc,ac,pc)

buildpb_modop7  /Users/brian/Downloads/punc/src/pmg/src_f77/buildPd.f:      subroutine buildPb_modop7 (nxf,nyf,nzf,nxc,nyc,nzc,

buildp_modop27  /Users/brian/Downloads/punc/src/pmg/src_f77/buildPd.f:      subroutine buildP_modop27 (nxf,nyf,nzf,nxc,nyc,nzc,ipc,rpc,ac,pc)

buildpb_modop27 /Users/brian/Downloads/punc/src/pmg/src_f77/buildPd.f:      subroutine buildPb_modop27 (nxf,nyf,nzf,nxc,nyc,nzc,

cghs    /Users/brian/Downloads/punc/src/pmg/src_f77/cgd.f:      subroutine cghs(nx,ny,nz,ipc,rpc,ac,cc,fc,x,p,ap,r,

fcgmg   /Users/brian/Downloads/punc/src/pmg/src_f77/cgmgd.f:      subroutine fcgmg(nx,ny,nz,x,iz,w0,w1,w2,w3,

cgmg    /Users/brian/Downloads/punc/src/pmg/src_f77/cgmgd.f:      subroutine cgmg(nx,ny,nz,x,iz,w0,w1,w2,w3,

getpre  /Users/brian/Downloads/punc/src/pmg/src_f77/cgmgd.f:      subroutine getpre(nx,ny,nz,iz,lev,nlev_real,r,pc)

cgmgdriv        /Users/brian/Downloads/punc/src/pmg/src_f77/cgmgdrvd.f:      subroutine cgmgdriv(iparm,rparm,iwork,rwork,u,

cgmgdriv2       /Users/brian/Downloads/punc/src/pmg/src_f77/cgmgdrvd.f:      subroutine cgmgdriv2(iparm,rparm,nx,ny,nz,u,iz,

delget
gsrb    /Users/brian/Downloads/punc/src/pmg/src_f77/gsd.f:      subroutine gsrb(nx,ny,nz,ipc,rpc,ac,cc,fc,x,w1,w2,r,

gsrb7   /Users/brian/Downloads/punc/src/pmg/src_f77/gsd.f:      subroutine gsrb7(nx,ny,nz,ipc,rpc,

gsrb27  /Users/brian/Downloads/punc/src/pmg/src_f77/gsd.f:      subroutine gsrb27(nx,ny,nz,ipc,rpc,

gsrb7x  /Users/brian/Downloads/punc/src/pmg/src_f77/gsd.f:      subroutine gsrb7X(nx,ny,nz,ipc,rpc,

readit
packmg
writit
matvec  /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine matvec(nx,ny,nz,ipc,rpc,ac,cc,x,y)

matvec7 /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine matvec7(nx,ny,nz,ipc,rpc,ac,cc,x,y)

matvec7_1s      /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine matvec7_1s(nx,ny,nz,ipc,rpc,oC,cc,oE,oN,uC,x,y)

matvec27        /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine matvec27(nx,ny,nz,ipc,rpc,ac,cc,x,y)

matvec27_1s     /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine matvec27_1s(nx,ny,nz,ipc,rpc,oC,cc,

mresid  /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine mresid(nx,ny,nz,ipc,rpc,ac,cc,fc,x,r)

mresid7 /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine mresid7(nx,ny,nz,ipc,rpc,ac,cc,fc,x,r)

mresid7_1s      /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine mresid7_1s(nx,ny,nz,ipc,rpc,oC,cc,fc,oE,oN,uC,x,r)

mresid27        /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine mresid27(nx,ny,nz,ipc,rpc,ac,cc,fc,x,r)

mresid27_1s     /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine mresid27_1s(nx,ny,nz,ipc,rpc,oC,cc,fc,

nmatvec /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine nmatvec(nx,ny,nz,ipc,rpc,ac,cc,x,y,w1)

nmatvec7        /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine nmatvec7(nx,ny,nz,ipc,rpc,ac,cc,x,y,w1)

nmatvecd7_1s    /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine nmatvecd7_1s(nx,ny,nz,ipc,rpc,

nmatvec27       /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine nmatvec27(nx,ny,nz,ipc,rpc,ac,cc,x,y,w1)

nmatvecd27_1s   /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine nmatvecd27_1s(nx,ny,nz,ipc,rpc,

nmresid /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine nmresid(nx,ny,nz,ipc,rpc,ac,cc,fc,x,r,w1)

nmresid7        /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine nmresid7(nx,ny,nz,ipc,rpc,ac,cc,fc,x,r,w1)

nmresid7_1s     /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine nmresid7_1s(nx,ny,nz,ipc,rpc,

nmresid27       /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine nmresid27(nx,ny,nz,ipc,rpc,ac,cc,fc,x,r,w1)

nmresid27_1s    /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine nmresid27_1s(nx,ny,nz,ipc,rpc,

interp  /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine interp(nxc,nyc,nzc,nxf,nyf,nzf,xin,xout,pc)

interp2 /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine interp2(nxc,nyc,nzc,nxf,nyf,nzf,xin,xout,

restrc  /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine restrc(nxf,nyf,nzf,nxc,nyc,nzc,xin,xout,pc)

restrc2 /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine restrc2(nxf,nyf,nzf,nxc,nyc,nzc,xin,xout,

extrac  /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine extrac(nxf,nyf,nzf,nxc,nyc,nzc,xin,xout)

interpz /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine interpZ(nxc,nyc,nzc,nxf,nyf,nzf,xin,xout)

restrcz /Users/brian/Downloads/punc/src/pmg/src_f77/matvecd.f:      subroutine restrcZ(nxf,nyf,nzf,nxc,nyc,nzc,xin,xout)

daxpy   /Users/brian/Downloads/punc/src/pmg/src_f77/mblasd.f:      subroutine daxpy(n,alpha,x,istep,y,jstep)

dcopy   /Users/brian/Downloads/punc/src/pmg/src_f77/mblasd.f:      subroutine dcopy(n,x,istep,y,jstep)

dasum   /Users/brian/Downloads/punc/src/pmg/src_f77/mblasd.f:      function dasum(n,x,istep)

dnrm1   /Users/brian/Downloads/punc/src/pmg/src_f77/mblasd.f:      function dnrm1(n,x,istep)

dnrm2   /Users/brian/Downloads/punc/src/pmg/src_f77/mblasd.f:      function dnrm2(n,x,istep)

dnrm8   /Users/brian/Downloads/punc/src/pmg/src_f77/mblasd.f:      function dnrm8(n,x,istep)

dscal   /Users/brian/Downloads/punc/src/pmg/src_f77/mblasd.f:      subroutine dscal(n,fac,x,istep)

ddot    /Users/brian/Downloads/punc/src/pmg/src_f77/mblasd.f:      function ddot(n,x,istep,y,jstep)

idamax  /Users/brian/Downloads/punc/src/pmg/src_f77/mblasd.f:      function idamax(n,sx,incx)

fmvcs   /Users/brian/Downloads/punc/src/pmg/src_f77/mgcsd.f:      subroutine fmvcs(nx,ny,nz,x,iz,w0,w1,w2,w3,

mvcs    /Users/brian/Downloads/punc/src/pmg/src_f77/mgcsd.f:      subroutine mvcs(nx,ny,nz,x,iz,w0,w1,w2,w3,

mgdriv  /Users/brian/Downloads/punc/src/pmg/src_f77/mgdrvd.f:      subroutine mgdriv(iparm,rparm,iwork,rwork,u,

mgdriv2 /Users/brian/Downloads/punc/src/pmg/src_f77/mgdrvd.f:      subroutine mgdriv2(iparm,rparm,nx,ny,nz,u,iz,

mgsz    /Users/brian/Downloads/punc/src/pmg/src_f77/mgdrvd.f:      subroutine mgsz(mgcoar,mgdisc,mgsolv,nx,ny,nz,nlev,nxc,nyc,nzc,

mgsize  /Users/brian/Downloads/punc/src/pmg/src_f77/mgdrvd.f:      subroutine mgsize(mgcoar,mgdisc,mgsolv,nx,ny,nz,nlev)

fmvfas  /Users/brian/Downloads/punc/src/pmg/src_f77/mgfasd.f:      subroutine fmvfas(nx,ny,nz,x,iz,w0,w1,w2,w3,w4,

mvfas   /Users/brian/Downloads/punc/src/pmg/src_f77/mgfasd.f:      subroutine mvfas(nx,ny,nz,x,iz,w0,w1,w2,w3,w4,

maxlev  /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      integer function maxlev(n1,n2,n3)

mkcors  /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine mkcors(numlev,nxold,nyold,nzold,nxnew,nynew,nznew)

corsr   /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine corsr(nold,nnew)

mkfine  /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine mkfine(numlev,nxold,nyold,nzold,nxnew,nynew,nznew)

finer   /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine finer(nold,nnew)

ivariv
prtini  /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine prtini(istop)

prtstp  /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine prtstp(iok,iters,rsnrm,rsden,orsnrm)

buildstr        /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine buildstr (nx,ny,nz,nlev,iz)

buildops        /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine buildops (nx,ny,nz,nlev,ipkey,iinfo,ido,iz,

buildcopy0      /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine buildcopy0 (nx,ny,nz,nxf,nyf,nzf,

buildharm0      /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine buildharm0 (nx,ny,nz,nxf,nyf,nzf,

buildgaler0     /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine buildgaler0 (nxf,nyf,nzf,nxc,nyc,nzc,ipkey,numdia,

buildgaler1     /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine buildgaler1 (nxf,nyf,nzf,nxc,nyc,nzc,numdia,

buildalg        /Users/brian/Downloads/punc/src/pmg/src_f77/mgsubd.f:      subroutine buildALG (nx,ny,nz,mode,nlev,iz,

epsmac
tstart  /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine tstart(before,overhd)

tstop   /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine tstop (before,overhd,cputme)

xaxpy   /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine xaxpy(nx,ny,nz,alpha,x,y)

xcopy   /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine xcopy(nx,ny,nz,x,y)

xnrm1   /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      function xnrm1(nx,ny,nz,x)

xnrm2   /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      function xnrm2(nx,ny,nz,x)

xnrm8   /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      function xnrm8(nx,ny,nz,x)

xscal   /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine xscal(nx,ny,nz,fac,x)

xdot    /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      function xdot(nx,ny,nz,x,y)

xdot3   /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      function xdot3(nx,ny,nz,x,y)

zeros   /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine zeros(nx,ny,nz,x)

xrand   /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine xrand(nx,ny,nz,x)

cinit   /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine cinit(nx,ny,nz,x,value)

azeros  /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine azeros(nx,ny,nz,x)

axrand  /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine axrand(nx,ny,nz,x)

xcopy_small     /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine xcopy_small(nx,ny,nz,x,y)

xcopy_large     /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine xcopy_large(nx,ny,nz,x,y)

fbound  /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine fbound(ibound,nx,ny,nz,x,gxc,gyc,gzc)

fbound00        /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine fbound00(nx,ny,nz,x)

xprint  /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine xprint(nx,ny,nz,x)

prtmatd /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine prtmatd(nx,ny,nz,ipc,rpc,ac)

prtmatd7        /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine prtmatd7(nx,ny,nz,ipc,rpc,oC,oE,oN,uC)

prtmatd27       /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine prtmatd27(nx,ny,nz,ipc,rpc,oC,oE,oN,uC,

prtmatb /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine prtmatb(a,n,m,lda)

linesearch      /Users/brian/Downloads/punc/src/pmg/src_f77/mikpckd.f:      subroutine linesearch(nx,ny,nz,alpha,

dpbco   /Users/brian/Downloads/punc/src/pmg/src_f77/mlinpckd.f:      SUBROUTINE DPBCO(ABD,LDA,N,M,RCOND,Z,INFO)

dpbfa   /Users/brian/Downloads/punc/src/pmg/src_f77/mlinpckd.f:      SUBROUTINE DPBFA(ABD,LDA,N,M,INFO)

dpbsl   /Users/brian/Downloads/punc/src/pmg/src_f77/mlinpckd.f:      SUBROUTINE DPBSL(ABD,LDA,N,M,B)

ncghs   /Users/brian/Downloads/punc/src/pmg/src_f77/ncgd.f:      subroutine ncghs(nx,ny,nz,ipc,rpc,ac,cc,fc,x,p,ap,r,zk,zkp1,tmp,

ncghsdriv       /Users/brian/Downloads/punc/src/pmg/src_f77/ncgdrvd.f:      subroutine ncghsdriv(iparm,rparm,iwork,rwork,u,

ncghsgo /Users/brian/Downloads/punc/src/pmg/src_f77/ncgdrvd.f:      subroutine ncghsgo(nx,ny,nz,x,r,p,ap,zk,zkp1,tmp,

cghsgo  /Users/brian/Downloads/punc/src/pmg/src_f77/ncgdrvd.f:      subroutine cghsgo(nx,ny,nz,x,r,p,ap,zk,zkp1,tmp,

newdriv /Users/brian/Downloads/punc/src/pmg/src_f77/newdrvd.f:      subroutine newdriv(iparm,rparm,iwork,rwork,u,

newdriv2        /Users/brian/Downloads/punc/src/pmg/src_f77/newdrvd.f:      subroutine newdriv2(iparm,rparm,nx,ny,nz,u,iz,

fnewton /Users/brian/Downloads/punc/src/pmg/src_f77/newtond.f:      subroutine fnewton(nx,ny,nz,x,iz,w0,w1,w2,w3,

newton  /Users/brian/Downloads/punc/src/pmg/src_f77/newtond.f:      subroutine newton(nx,ny,nz,x,iz,w0,w1,w2,w3,

getjac  /Users/brian/Downloads/punc/src/pmg/src_f77/newtond.f:      subroutine getjac (nx,ny,nz,nlev_real,iz,lev,ipkey,

ngsrb   /Users/brian/Downloads/punc/src/pmg/src_f77/ngsd.f:      subroutine ngsrb(nx,ny,nz,ipc,rpc,ac,cc,fc,x,w1,w2,r,

ngsrb7  /Users/brian/Downloads/punc/src/pmg/src_f77/ngsd.f:      subroutine ngsrb7(nx,ny,nz,ipc,rpc,

ngsrb27 /Users/brian/Downloads/punc/src/pmg/src_f77/ngsd.f:      subroutine ngsrb27(nx,ny,nz,ipc,rpc,

ngsrb7x /Users/brian/Downloads/punc/src/pmg/src_f77/ngsd.f:      subroutine ngsrb7X(nx,ny,nz,ipc,rpc,

ngsrbdriv       /Users/brian/Downloads/punc/src/pmg/src_f77/ngsdrvd.f:      subroutine ngsrbdriv(iparm,rparm,iwork,rwork,u,

ngsrbdriv2      /Users/brian/Downloads/punc/src/pmg/src_f77/ngsdrvd.f:      subroutine ngsrbdriv2(iparm,rparm,nx,ny,nz,u,iz,

ngsrbgo /Users/brian/Downloads/punc/src/pmg/src_f77/ngsdrvd.f:      subroutine ngsrbgo(nx,ny,nz,x,r,w1,w2,

gsrbgo  /Users/brian/Downloads/punc/src/pmg/src_f77/ngsdrvd.f:      subroutine gsrbgo(nx,ny,nz,x,r,w1,w2,

ninterp /Users/brian/Downloads/punc/src/pmg/src_f77/ninterpd.f:      subroutine ninterp(nxc,nyc,nzc,nxf,nyf,nzf,xin,xout,pc,

ninterp7        /Users/brian/Downloads/punc/src/pmg/src_f77/ninterpd.f:      subroutine ninterp7(nxc,nyc,nzc,nxf,nyf,nzf,xin,xout,

ninterp27       /Users/brian/Downloads/punc/src/pmg/src_f77/ninterpd.f:      subroutine ninterp27(nxc,nyc,nzc,nxf,nyf,nzf,xin,xout,

nrich   /Users/brian/Downloads/punc/src/pmg/src_f77/nrchd.f:      subroutine nrich(nx,ny,nz,ipc,rpc,ac,cc,fc,x,w1,w2,r,

nrich7  /Users/brian/Downloads/punc/src/pmg/src_f77/nrchd.f:      subroutine nrich7(nx,ny,nz,ipc,rpc,

nrich27 /Users/brian/Downloads/punc/src/pmg/src_f77/nrchd.f:      subroutine nrich27(nx,ny,nz,ipc,rpc,

nrichdriv       /Users/brian/Downloads/punc/src/pmg/src_f77/nrchdrvd.f:      subroutine nrichdriv(iparm,rparm,iwork,rwork,u,

nrichdriv2      /Users/brian/Downloads/punc/src/pmg/src_f77/nrchdrvd.f:      subroutine nrichdriv2(iparm,rparm,nx,ny,nz,u,iz,

nrichgo /Users/brian/Downloads/punc/src/pmg/src_f77/nrchdrvd.f:      subroutine nrichgo(nx,ny,nz,x,r,w1,w2,

richgo  /Users/brian/Downloads/punc/src/pmg/src_f77/nrchdrvd.f:      subroutine richgo(nx,ny,nz,x,r,w1,w2,

nsmooth /Users/brian/Downloads/punc/src/pmg/src_f77/nsmoothd.f:      subroutine nsmooth(nx,ny,nz,ipc,rpc,ac,cc,fc,x,w1,w2,r,

nsor    /Users/brian/Downloads/punc/src/pmg/src_f77/nsord.f:      subroutine nsor(nx,ny,nz,ipc,rpc,ac,cc,fc,x,w1,w2,r,

nsor7   /Users/brian/Downloads/punc/src/pmg/src_f77/nsord.f:      subroutine nsor7(nx,ny,nz,ipc,rpc,

nsor27  /Users/brian/Downloads/punc/src/pmg/src_f77/nsord.f:      subroutine nsor27(nx,ny,nz,ipc,rpc,

nsordriv        /Users/brian/Downloads/punc/src/pmg/src_f77/nsordrvd.f:      subroutine nsordriv(iparm,rparm,iwork,rwork,u,

nsordriv2       /Users/brian/Downloads/punc/src/pmg/src_f77/nsordrvd.f:      subroutine nsordriv2(iparm,rparm,nx,ny,nz,u,iz,

nsorgo  /Users/brian/Downloads/punc/src/pmg/src_f77/nsordrvd.f:      subroutine nsorgo(nx,ny,nz,x,r,w1,w2,

sorgo   /Users/brian/Downloads/punc/src/pmg/src_f77/nsordrvd.f:      subroutine sorgo(nx,ny,nz,x,r,w1,w2,

nwjac   /Users/brian/Downloads/punc/src/pmg/src_f77/nwjd.f:      subroutine nwjac(nx,ny,nz,ipc,rpc,ac,cc,fc,x,w1,w2,r,

nwjac7  /Users/brian/Downloads/punc/src/pmg/src_f77/nwjd.f:      subroutine nwjac7(nx,ny,nz,ipc,rpc,

nwjac27 /Users/brian/Downloads/punc/src/pmg/src_f77/nwjd.f:      subroutine nwjac27(nx,ny,nz,ipc,rpc,

nwjacdriv       /Users/brian/Downloads/punc/src/pmg/src_f77/nwjdrvd.f:      subroutine nwjacdriv(iparm,rparm,iwork,rwork,u,

nwjacdriv2      /Users/brian/Downloads/punc/src/pmg/src_f77/nwjdrvd.f:      subroutine nwjacdriv2(iparm,rparm,nx,ny,nz,u,iz,

nwjacgo /Users/brian/Downloads/punc/src/pmg/src_f77/nwjdrvd.f:      subroutine nwjacgo(nx,ny,nz,x,r,w1,w2,

wjacgo  /Users/brian/Downloads/punc/src/pmg/src_f77/nwjdrvd.f:      subroutine wjacgo(nx,ny,nz,x,r,w1,w2,

c_scal
dc_scal
c_vec
dc_vec
fillco
power   /Users/brian/Downloads/punc/src/pmg/src_f77/powerd.f:      subroutine power(nx,ny,nz,iz,ilev,ipc,rpc,ac,cc,

ipower  /Users/brian/Downloads/punc/src/pmg/src_f77/powerd.f:      subroutine ipower(nx,ny,nz,u,iz,w0,w1,w2,w3,w4,

mpower  /Users/brian/Downloads/punc/src/pmg/src_f77/powerd.f:      subroutine mpower(nx,ny,nz,u,iz,w0,w1,w2,w3,w4,

rich    /Users/brian/Downloads/punc/src/pmg/src_f77/rchd.f:      subroutine rich(nx,ny,nz,ipc,rpc,ac,cc,fc,x,w1,w2,r,

rich7   /Users/brian/Downloads/punc/src/pmg/src_f77/rchd.f:      subroutine rich7(nx,ny,nz,ipc,rpc,

rich27  /Users/brian/Downloads/punc/src/pmg/src_f77/rchd.f:      subroutine rich27(nx,ny,nz,ipc,rpc,

smooth  /Users/brian/Downloads/punc/src/pmg/src_f77/smoothd.f:      subroutine smooth(nx,ny,nz,ipc,rpc,ac,cc,fc,x,w1,w2,r,

sor     /Users/brian/Downloads/punc/src/pmg/src_f77/sord.f:      subroutine sor(nx,ny,nz,ipc,rpc,ac,cc,fc,x,w1,w2,r,

sor7    /Users/brian/Downloads/punc/src/pmg/src_f77/sord.f:      subroutine sor7(nx,ny,nz,ipc,rpc,

sor27   /Users/brian/Downloads/punc/src/pmg/src_f77/sord.f:      subroutine sor27(nx,ny,nz,ipc,rpc,

wjac    /Users/brian/Downloads/punc/src/pmg/src_f77/wjd.f:      subroutine wjac(nx,ny,nz,ipc,rpc,ac,cc,fc,x,w1,w2,r,

wjac7   /Users/brian/Downloads/punc/src/pmg/src_f77/wjd.f:      subroutine wjac7(nx,ny,nz,ipc,rpc,

wjac27  /Users/brian/Downloads/punc/src/pmg/src_f77/wjd.f:      subroutine wjac27(nx,ny,nz,ipc,rpc,
nsoblath commented 2 years ago

@BJWiley233 Unfortunately I've been unable to replicate the problems you were seeing. I don't have a Mac running macOS 12, but GitHub Actions offers runners using that OS version, so I did a test there. I added the additional build in 84f22e27651b9863bffb6a6af1df91bcf32055db, and made no other changes to the build. You can find the log of the build here.

There are two potential differences, as far as I can tell, from when you ran your tests:

  1. There may have been XCode/Clang/LLVM/etc version updates since early March. The Clang version used in the build is reported to be AppleClang 13.1.6.13160021.
  2. I used an updated version of FETK from the one you were using. I started from the latest commit on branch nsoblath/issue_62 (86e074ecfa19b818e1896ea44082d14bc7e44082), which itself has a few changes made compared to v1.9.2.
BJWiley233 commented 2 years ago

Thanks @nsoblath,

You wouldn't see the issue unless you were trying to make shared libaries for OpenMP compilation. The issue is that these 4 functions below are defined in file punc/tools/tests/pmg/pdef.f and you would need to run a f2c script on this file to include in the punc/src/pmg folder. I am surprised Michael didn't catch this :)

c_scal dc_scal c_vec dc_vec

We also need to include rand.c and secd.c for rand and tsecnd C implementations from Fortran.

nsoblath commented 2 years ago

@BJWiley233 Thanks for pointing this out and summarizing the conclusion from your investigation. We'll proceed with a new release of FETK for now, since I want to complete a release of APBS with several bug fixes and small updates before tackling the OpenMP compilation.