search

xsdk-project / xsdk-examples

Example codes demonstrating the use of various XSDK packages in combination.
17 stars 10 forks source link

xsdk-examples@0.3.0%gcc@9.2.0+cuda build failure at UTK #8

Closed balay closed 2 years ago

balay commented 2 years ago 
/usr/bin/ld: CMakeFiles/advection.dir/advection.cpp.o: undefined reference to symbol 'cusparseDestroy@@libcusparse.so.11'
/nfs/apps/spacks/2021-02-09/opt/spack/linux-centos7-x86_64/gcc-7.3.0/cuda-11.2.0-ikruu5mo6dtt3avvmwsejouqhxu4btdm/lib64/libcusparse.so.11: error adding symbols: DSO missing from command line

spack-build-out.txt

Ref: ./bin/spack install --fail-fast -j24 xsdk-examples+cuda ^xsdk cuda_arch=70 ^cuda@11.2.0 ^openmpi ^mfem+shared

Don't remember why I had to use mfem+shared - I'll have to retry that

balay commented 2 years ago

Here is one with ./bin/spack install --fail-fast -j24 xsdk-examples+cuda ^xsdk cuda_arch=70 ^cuda@11.2.0 ^openmpi

Slightly different errors:


/usr/bin/ld: /data/balay/spack.x/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/mfem-4.3.0-5e4ws4okdqlrglpujaqr47t4drgiluh2/lib/libmfem.a(mem_manager.o): undefined reference to symbol 'cudaStreamSynchronize@@libcudart.so.11.0'

[spack-build-out.txt](https://github.com/xsdk-project/xsdk-examples/files/8180163/spack-build-out.txt)
balos1 commented 2 years ago

@balay This should be fixed on the balos1/updates branch. I don't think mfem+shared is needed.

balay commented 2 years ago

@balos1 I attempted a rebuld with the updated branch. I don't think the build went that far this time. Attaching the log

spack-build-out.txt

Ref:

./bin/spack install --fail-fast -j24 xsdk-examples+cuda ^xsdk cuda_arch=70 ^cuda@11.2.0 ^openmpi ^mfem+shared

balay@xsdk:/data/balay/spack/spack-stage/spack-stage-xsdk-examples-0.3.0-kw7obfvl7ymj76fkvbjbxbljgwexnoww/spack-src>git log
commit e6657b1c9077bafac6a20425a378f0b6153a4c72 (grafted, HEAD -> balos1/updates, origin/balos1/updates)
keitat commented 2 years ago

Failed at MFEM. Do you have any ideas?

==> Installing mfem-4.2.0-5xzjopnrthwhzn7djooawcprjthslgb5
==> No binary for mfem-4.2.0-5xzjopnrthwhzn7djooawcprjthslgb5 found: installing from source
==> Using cached archive: /home/knteran/spack/var/spack/cache/_source-cache/archive/43/4352a225b55948d2e73a5ee88cece0e88bdbe7ba6726a23d68b2736d3221a86d.tar.gz
==> No patches needed for mfem
==> mfem: Executing phase: 'configure'
==> mfem: Executing phase: 'build'
==> mfem: Executing phase: 'install'
==> Error: ProcessError: Command exited with status 2:
    'make' 'install'

102 errors found in build log:
     5498    general/optparser.o:98:16: warning: null character(s) ignored
     5499    general/optparser.o:98:32: warning: null character(s) ignored
     5500    general/optparser.o:98:36: warning: null character(s) ignored
     5501    general/optparser.o:98:57: warning: null character(s) ignored
     5502    general/optparser.o:98:65: warning: null character(s) ignored
     5503    general/optparser.o:98:80: warning: null character(s) ignored
  >> 5504    general/optparser.o(1): error: unrecognized token
     5505    
  >> 5506    general/optparser.o(1): error: expected a declaration
     5507    
  >> 5508    general/optparser.o(1): error: unrecognized token
     5509    
  >> 5510    general/optparser.o(1): error: unrecognized token
     5511    
  >> 5512    general/optparser.o(1): error: unrecognized token
     5513    
  >> 5514    general/optparser.o(1): error: unrecognized token
     5515    
  >> 5516    general/optparser.o(1): error: unrecognized token
     5517    
  >> 5518    general/optparser.o(1): error: unrecognized token
     5519    
  >> 5520    general/optparser.o(1): error: unrecognized token
     5521    
  >> 5522    general/optparser.o(1): error: unrecognized token
     5523    
  >> 5524    general/optparser.o(1): error: unrecognized token
     5525    
  >> 5526    general/optparser.o(1): error: unrecognized token
     5527    
  >> 5528    general/optparser.o(1): error: unrecognized token
     5529    
  >> 5530    general/optparser.o(1): error: unrecognized token
     5531    
  >> 5532    general/optparser.o(1): error: unrecognized token
     5533    
  >> 5534    general/optparser.o(1): error: unrecognized token
     5535    
  >> 5536    general/optparser.o(1): error: unrecognized token
     5537    
  >> 5538    general/optparser.o(1): error: unrecognized token
     5539    
  >> 5540    general/optparser.o(1): error: unrecognized token
     5541    
  >> 5542    general/optparser.o(1): error: unrecognized token
     5543    
  >> 5544    general/optparser.o(1): error: unrecognized token
     5545    
  >> 5546    general/optparser.o(1): error: unrecognized token
     5547    
  >> 5548    general/optparser.o(1): error: unrecognized token
     5549    
  >> 5550    general/optparser.o(1): error: unrecognized token
     5551    
  >> 5552    general/optparser.o(1): error: unrecognized token
     5553    
  >> 5554    general/optparser.o(1): error: unrecognized token
     5555    
  >> 5556    general/optparser.o(1): error: unrecognized token
     5557    
  >> 5558    general/optparser.o(1): error: unrecognized token
     5559    
  >> 5560    general/optparser.o(1): error: unrecognized token
     5561    
  >> 5562    general/optparser.o(1): error: unrecognized token
     5563    
  >> 5564    general/optparser.o(1): error: unrecognized token
     5565    
  >> 5566    general/optparser.o(1): error: unrecognized token
     5567    
  >> 5568    general/optparser.o(1): error: unrecognized token
     5569    
  >> 5570    general/optparser.o(1): error: unrecognized token
     5571    
  >> 5572    general/optparser.o(1): error: unrecognized token
     5573    
  >> 5574    general/optparser.o(1): error: unrecognized token
     5575    
  >> 5576    general/optparser.o(1): error: unrecognized token
     5577    
  >> 5578    general/optparser.o(1): error: unrecognized token
     5579    
  >> 5580    general/optparser.o(1): error: unrecognized token
     5581    
  >> 5582    general/optparser.o(1): error: unrecognized token
     5583    
  >> 5584    general/optparser.o(1): error: unrecognized token
     5585    
  >> 5586    general/optparser.o(1): error: unrecognized token
     5587    
  >> 5588    general/optparser.o(1): error: unrecognized token
     5589    
  >> 5590    general/optparser.o(1): error: unrecognized token
     5591    
  >> 5592    general/optparser.o(1): error: unrecognized token
     5593    
  >> 5594    general/optparser.o(1): error: unrecognized token
     5595    
  >> 5596    general/optparser.o(1): error: unrecognized token
     5597    
  >> 5598    general/optparser.o(1): error: unrecognized token
     5599    
  >> 5600    general/optparser.o(1): error: unrecognized token
     5601    
  >> 5602    general/optparser.o(1): error: unrecognized token
     5603    
  >> 5604    general/optparser.o(1): error: unrecognized token
     5605    
  >> 5606    general/optparser.o(1): error: unrecognized token
     5607    
  >> 5608    general/optparser.o(1): error: unrecognized token
     5609    
  >> 5610    general/optparser.o(1): error: unrecognized token
     5611    
  >> 5612    general/optparser.o(1): error: unrecognized token
     5613    
  >> 5614    general/optparser.o(1): error: unrecognized token
     5615    
  >> 5616    general/optparser.o(1): error: unrecognized token
     5617    
  >> 5618    general/optparser.o(1): error: unrecognized token
     5619    
  >> 5620    general/optparser.o(1): error: unrecognized token
     5621    
  >> 5622    general/optparser.o(1): error: unrecognized token
     5623    
  >> 5624    general/optparser.o(1): error: unrecognized token
     5625    
  >> 5626    general/optparser.o(1): error: unrecognized token
     5627    
  >> 5628    general/optparser.o(1): error: unrecognized token
     5629    
  >> 5630    general/optparser.o(1): error: unrecognized token
     5631    
  >> 5632    general/optparser.o(1): error: unrecognized token
     5633    
  >> 5634    general/optparser.o(1): error: unrecognized token
     5635    
  >> 5636    general/optparser.o(1): error: unrecognized token
     5637    
  >> 5638    general/optparser.o(1): error: unrecognized token
     5639    
  >> 5640    general/optparser.o(1): error: unrecognized token
     5641    
  >> 5642    general/optparser.o(1): error: unrecognized token
     5643    
  >> 5644    general/optparser.o(1): error: unrecognized token
     5645    
  >> 5646    general/optparser.o(1): error: unrecognized token
     5647    
  >> 5648    general/optparser.o(1): error: unrecognized token
     5649    
  >> 5650    general/optparser.o(1): error: unrecognized token
     5651    
  >> 5652    general/optparser.o(1): error: unrecognized token
     5653    
  >> 5654    general/optparser.o(1): error: unrecognized token
     5655    
  >> 5656    general/optparser.o(1): error: unrecognized token
     5657    
  >> 5658    general/optparser.o(1): error: unrecognized token
     5659    
  >> 5660    general/optparser.o(1): error: unrecognized token
     5661    
  >> 5662    general/optparser.o(1): error: unrecognized token
     5663    
  >> 5664    general/optparser.o(1): error: unrecognized token
     5665    
  >> 5666    general/optparser.o(1): error: unrecognized token
     5667    
  >> 5668    general/optparser.o(1): error: unrecognized token
     5669    
  >> 5670    general/optparser.o(1): error: unrecognized token
     5671    
  >> 5672    general/optparser.o(1): error: unrecognized token
     5673    
  >> 5674    general/optparser.o(1): error: unrecognized token
     5675    
  >> 5676    general/optparser.o(1): error: unrecognized token
     5677    
  >> 5678    general/optparser.o(1): error: unrecognized token
     5679    
  >> 5680    general/optparser.o(1): error: unrecognized token
     5681    
  >> 5682    general/optparser.o(1): error: unrecognized token
     5683    
  >> 5684    general/optparser.o(1): error: unrecognized token
     5685    
  >> 5686    general/optparser.o(1): error: unrecognized token
     5687    
  >> 5688    general/optparser.o(1): error: unrecognized token
     5689    
  >> 5690    general/optparser.o(1): error: unrecognized token
     5691    
  >> 5692    general/optparser.o(1): error: unrecognized token
     5693    
  >> 5694    general/optparser.o(1): error: unrecognized token
     5695    
  >> 5696    general/optparser.o(1): error: unrecognized token
     5697    
  >> 5698    general/optparser.o(1): error: unrecognized token
     5699    
  >> 5700    general/optparser.o(1): error: unrecognized token
     5701    
  >> 5702    general/optparser.o(1): error: unrecognized token
     5703    
  >> 5704    Error limit reached.
     5705    100 errors detected in the compilation of "general/optparser.o".
     5706    Compilation terminated.
  >> 5707    make: *** [install] Error 1

See build log for details:
  /tmp/knteran/spack-stage/spack-stage-mfem-4.2.0-5xzjopnrthwhzn7djooawcprjthslgb5/spack-build-out.txt
keitat commented 2 years ago

It seems that Spack is extremely sensitive to a little change in pacakges.yaml. The MFEM problem is gone.

keitat commented 2 years ago

The new problem with linking CUDA libraries...

/usr/bin/ld: /home/knteran/spack/opt/spack/linux-centos7-cascadelake/gcc-9
            .2.0/mfem-4.2.0-l5opsnhtuhtikkglgqv4h4wxljosygoc/lib/libmfem.a(mem_manager
            .o): undefined reference to symbol 'cudaStreamSynchronize@@@@libcudart.so.11
            .0'
balos1 commented 2 years ago

@keitat It looks like it is installing an old version of mfem, are you using the xsdk-examples branch of https://gitlab.com/xsdk-project/spack-xsdk for spack? xsdk@0.7.0 uses mfem@4.3.0.

balay commented 2 years ago

I've reattempted the cuda build with:

./bin/spack install --fail-fast -j24 xsdk-examples%gcc@9.2.0+cuda

And get:

CMake Error at CMakeLists.txt:66 (enable_language):
  No CMAKE_CUDA_COMPILER could be found.

  Tell CMake where to find the compiler by setting either the environment
  variable "CUDACXX" or the CMake cache entry CMAKE_CUDA_COMPILER to the full
  path to the compiler, or to the compiler name if it is in the PATH.

Perhaps it should somehow get this info from spack dependencies?

spack-build-out.txt

balos1 commented 2 years ago

@balay I believe the issue is that CUDA is enabled by default at https://github.com/xsdk-project/xsdk-examples/blob/c25cbae2a161a23a8a6ec453d1aef0863bd1f938/mfem/sundials/CMakeLists.txt#L5. I will push up a fix.

balos1 commented 2 years ago

@balay should be fixed by https://github.com/xsdk-project/xsdk-examples/commit/7609d1d9bc8c89e1ec9305c5373c17aafc2c8712

balay commented 2 years ago

@balos1 attaching failure from latest build.

spack-build-out.txt

balos1 commented 2 years ago

I just completed a successful build w/o Trilinos (will open a new issue about that) on Lassen with this commit from spack-xsdk and this commit from xsdk-examples.

[+] /usr/tce/packages/cmake/cmake-3.21.1 (external cmake-3.21.1-kj4qengp6v5nrsco5tg4dcgvn4w25byq)
==> cuda@11.2.0 : has external module in ['cuda/11.2.0']
[+] /usr/tce/packages/cuda/cuda-11.2.0 (external cuda-11.2.0-awwtnkwhetwpi3glul4od6bblvzq4wwg)
[+] /usr/tce/packages/spectrum-mpi/spectrum-mpi-rolling-release-gcc-8.3.1 (external spectrum-mpi-10.3.1.03rtm0-rm7mmctksbz6dqg5fopug73rrxd4jxt4)
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/gnuconfig-2021-08-14-cb4jv6vgyhe4s32xkrotmv43ruipclos
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/zlib-1.2.12-7xr6otkt5s76wxmvuigexzx2wflgugnx
[+] /usr/tce/packages/python/python-3.8.2 (external python-3.8.2-dxpf7cvfyw3q4gvnxvoeekkcgzafxpap)
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/eigen-3.4.0-e5bo4uzbdso2csa5j7xl426kt2rzuxfm
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/kokkos-3.6.00-qiivnyvwank6c23jyn572pmowm7qcnq7
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/zfp-0.5.5-7dmshdwv72tykjoebszwesjqzix43aur
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/metis-5.1.0-jblc7o42mrqnsm5enybflrsyhzm3y3dz
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/ginkgo-1.4.0-xid4t7jeidbstmwmiu7ne3olynvv37js
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/pumi-2.2.6-gd6acbex3l3y3gb7ohonhgx5ld76n4xg
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/pkgconf-1.8.0-buqziuzdaboaggycopntpzwdqhkburr6
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/berkeley-db-18.1.40-criskmc233pm6k5ctxmitgwnncwpdqyy
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/fftw-3.3.10-wrviaybfljcnvebsyghmxqq4jio3rwy2
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/xz-5.2.5-zrfyybeti7lixvjayxqry66a4ns5smvc
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/libiconv-1.16-id5xdfyf7l6dopl7fenx7zbrnrv2y3xx
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/omega-h-9.34.1-gez3pnd42ui6rqgb5ikpqaz5qpnvzo5s
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/py-pip-21.3.1-fjezclxkd6cvsujmtl5vhic5nglyel3k
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/arborx-1.1-up2wk22gbpgxwhpoysepuxccs7hfu26u
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/parmetis-4.0.3-6z6f6hba4o6pxkxhm2timo4uwb3wgpxy
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/ncurses-6.2-yardclcyy6rkl3ny3vlpklxdetbkovhz
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/hdf5-1.12.2-cqwfto2pmi5czngiobx56ghg7kavv3pq
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/diffutils-3.8-2sdcdoees5fxsjlt4aku4d3fqdnoie4i
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/libxml2-2.9.13-7v45tlo3ccnkb6zpcogda34egljue74z
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/py-wheel-0.37.0-xlmlmhhstscj5ojibyi74osfdzzz2maz
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/readline-8.1-6mu2m4pjxhup7cfembwyvhjxk6mgvb6s
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/bzip2-1.0.8-4akxbe436i3f7pbuuwj55gyua3zzisao
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/py-setuptools-59.4.0-2rezvczk4chwcpc7q24sjrgrbj2lrbux
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/gdbm-1.19-s4ezwp4dmbjg6q2xhglorf4iriaczsqk
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/boost-1.78.0-g73p7itjloe3bu42qwriub6o3ynp72tz
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/py-psutil-5.8.0-mve4jc7bjvfp64kz3pcvkd3h7m7h6rkg
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/py-cython-0.29.24-ynla5lfwzuu7qycff3px5j45t74tv5fz
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/py-mpi4py-3.1.2-zd7vocanpvbplxmzzfk7k4566rgpvtqd
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/perl-5.34.1-wihezq6mcnofyybgcqex2who7nrnt7jm
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/openblas-0.3.20-xpxbrf6tzagw77h366goov2vlwersbih
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/arpack-ng-3.8.0-bv6qy3x7yeozsaf5tkqmw7hrkiscbc76
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/plasma-21.8.29-gw2gmvyhgxxgrtijwqlwvmzgmu4hfj76
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/blaspp-2021.04.01-tymkyra4do46wb6zozzcnwvbgqkoludf
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/netlib-scalapack-2.2.0-4pnrzqd2ytordlaaaflunfp7c3xdcxyp
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/py-numpy-1.19.5-3yamnekaujmfogmrb5xmufzsajdo5i25
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/magma-2.6.1-juuflwedijztdwpqixgjuaapnrc5uja7
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/superlu-dist-7.1.1-7i2e4br2n6d5vikw7fx3646b3umv7ctu
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/lapackpp-2021.04.00-rck5h63ymnj4gyubhgbhqxf4ktznippv
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/butterflypack-2.0.0-srtmjffz5ewx7acxp5uexk56n4pgcm2b
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/tasmanian-7.7-fmgczrqml6m3kzngunp5ilpvsm5owwhi
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/heffte-2.2.0-2fa3v627xnm757ovsfryqtdccvbqgobp
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/hypre-2.23.0-myr2gobbxptrptallrnbagjqmqy4v5hu
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/slate-2021.05.02-zt4ipj6ohelpk6x4zhy7zmdsoyilpggw
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/strumpack-6.1.0-wr6kp3uvgzvhrziwhfqf7v4ddba5cczk
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/petsc-3.16.1-fdo6a2fitei37ci4ojouvv6e3nm73zbx
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/phist-1.9.5-7q4vcdfjvqhcqjfu272jatg6ynyevm5a
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/sundials-5.8.0-fo6hg5kau3ta4rxlx3tkyqafstoacjvf
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/py-petsc4py-3.16.1-ggqb5hvvdtidt2dhtppba4yeqj27szad
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/pflotran-3.0.2-cwfzpcibikgnbwovnv22yksc5uqo7zeo
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/slepc-3.16.0-d2fnkyusudop2ykoxlqkjrtn4aj3252m
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/precice-2.3.0-jlkggvroa3i3djlo4366thlbkeq4362y
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/mfem-4.3.0-h2jovzupzsoqcp5pogoxj526nz35zspu
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/amrex-21.10-43tvamx2e75tosh7hhprwnyomqfvdhn3
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/py-libensemble-0.8.0-eg5w6tz57k2ckyrgvph4lreiacoa7fy5
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/alquimia-1.0.9-fv6xpfupyldb65elvdkwwxnfbbm75qh2
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/xsdk-0.7.0-o7o5cdq47bobjbtumczj6q7ax3cpowky
==> Installing xsdk-examples-0.3.0-k252fnv4jon2ypiqn44k6kbffog4l2is
==> No binary for xsdk-examples-0.3.0-k252fnv4jon2ypiqn44k6kbffog4l2is found: installing from source
==> No patches needed for xsdk-examples
==> xsdk-examples: Executing phase: 'cmake'
==> xsdk-examples: Executing phase: 'build'
==> xsdk-examples: Executing phase: 'install'
==> xsdk-examples: Successfully installed xsdk-examples-0.3.0-k252fnv4jon2ypiqn44k6kbffog4l2is
  Fetch: 2.44s.  Build: 39.38s.  Total: 41.82s.
[+] /usr/WS2/balos1/xsdk/spack/opt/spack/linux-rhel7-power9le/gcc-8.3.1/xsdk-examples-0.3.0-k252fnv4jon2ypiqn44k6kbffog4l2is
==> Warning: Missing a source id for xsdk@0.7.0
==> Warning: Missing a source id for xsdk-examples@0.3.0
==> Updating view at /g/g13/balos1/Workspaces/xsdk/spack-lassen-view
==> Warning: Skipping external package: spectrum-mpi@10.3.1.03rtm0%gcc@8.3.1 arch=linux-rhel7-power9le/rm7mmct
==> Warning: Skipping external package: cuda@11.2.0%gcc@8.3.1~allow-unsupported-compilers~dev arch=linux-rhel7-power9le/awwtnkw
==> Warning: Skipping external package: cmake@3.21.1%gcc@8.3.1~doc+ncurses+ownlibs~qt build_type=Release arch=linux-rhel7-power9le/kj4qeng
==> Warning: Skipping external package: python@3.8.2%gcc@8.3.1+bz2+ctypes+dbm~debug+ensurepip+libxml2+lzma+nis~optimizations+pic+pyexpat~pythoncmd+readline+shared+sqlite3+ssl+tix+tkinter~ucs4+uuid+zlib patches=0d98e93,f2fd060 arch=linux-rhel7-power9le/dxpf7cv
==> Warning: Missing a source id for xsdk@0.7.0
==> Warning: Missing a source id for xsdk-examples@0.3.0

@luszczek @balay

spack.lock.txt spack.yaml.txt

balay commented 2 years ago

require amrex 22.04 for amrex+sundials examples

Since xsdk@0.7.0 has amrex@21.10 - presumably this example is skipped?

I still have failures with +cuda build. Attaching the log.

spack-build-out.txt

balay commented 2 years ago

Sorry - I had and issue with cuda spec. Also there is some issue with concrerizer passing in cuda options from xsdk-examples to xsdk. [perhaps xsdk-examples should be converted to use cudapackage?]

The following works for me.

./bin/spack install --fail-fast -j24 xsdk-examples%gcc@9.2.0+cuda ^xsdk+cuda cuda_arch=70 ^cuda@11.2.0 ^openmpi
balay@xsdk:/data/balay/spack.zz>/data/balay/spack.zz/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/openmpi-4.1.3-rcgotw4jfvtpbqhl2xy6tut2g6u6c3q5/bin/mpiexec -n 2 /data/balay/spack.zz/opt/spack/linux-centos7-cascadelake/gcc-9.2.0/xsdk-examples-0.3.0-nyfyz4mvqkmxonhkduu6nexftprzur7t/bin/ex19 -dm_vec_type cuda -dm_mat_type aijcusparse -da_refine 3 -snes_monitor_short -ksp_norm_type unpreconditioned -pc_type hypre
lid velocity = 0.0016, prandtl # = 1., grashof # = 1.
  0 SNES Function norm 0.0406612 
  1 SNES Function norm 3.3435e-06 
  2 SNES Function norm 2.611e-11 
Number of SNES iterations = 2
balos1 commented 2 years ago

require amrex 22.04 for amrex+sundials examples

Since xsdk@0.7.0 has amrex@21.10 - presumably this example is skipped?

Correct, see https://github.com/xsdk-project/xsdk-examples/blob/balos1/updates/amrex/CMakeLists.txt#L3