search

kokkos / kokkos-kernels

Kokkos C++ Performance Portability Programming Ecosystem: Math Kernels - Provides BLAS, Sparse BLAS and Graph Kernels
Other
305 stars 96 forks source link

Build failure on cascadelake with cuda backend #1298

Open s-sajid-ali opened 2 years ago

s-sajid-ali commented 2 years ago

Build failure for kokkos-kernels@3.5.00 using spack on cascadelake CPU with the CUDA backend. Build error is:

==> Installing kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t
==> No binary for kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/a0/a03a41a047d95f9f07cd1e1d30692afdb75b5c705ef524e19c1d02fe60ccf8d1.tar.gz
==> No patches needed for kokkos-kernels
==> kokkos-kernels: Executing phase: 'cmake'
==> kokkos-kernels: Executing phase: 'build'
==> Error: ProcessError: Command exited with status 2:
    'make'

9 errors found in build log:
     298    [ 40%] Building CXX object src/CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/gemm/Blas3_gemm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDAUVMSPACE.cpp.o
     299    cd /wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src && /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientif
            ic7-cascadelake/gcc-9.4.0/kokkos-nvcc-wrapper-3.2.00-geqoj5bmyrvej2s4tefwzvd7thcjk4zj/bin/nvcc_wrapper -DKOKKOS_DEPENDENCE -Dkokkoskernels_EXPORTS -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-
            kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src
             -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-s
            tage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spa
            ck-src/src/impl/tpls -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/blas -I/wclustre/accelsim/sajid/packages/spack/var/spack/
            stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/blas/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv
            7dz42lvog2t/spack-src/src/sparse -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/sparse/impl -I/wclustre/accelsim/sajid/packag
            es/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/graph -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawl
            qf43snj4qgc36lv7dz42lvog2t/spack-src/src/graph/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched -I/wclustre/accels
            im/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched/dense -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokko
            s-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched/dense/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src
            /src/batched/sparse -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched/sparse/impl -I/wclustre/accelsim/sajid/packages/sp
            ack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/common -isystem /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-9.4.0/kokkos-3.5
            .00-fk5fpcsh7o3ih7bkd3o6xvsikjr27mq3/include -O2 -g -DNDEBUG -fPIC -march=skylake-avx512 -mtune=skylake-avx512 -mrtm -expt-extended-lambda -Wext-lambda-captures-this -arch=sm_70 -std=c++14 -MD -MT src/CMakeFiles/kokkoskernel
            s.dir/impl/generated_specializations_cpp/gemm/Blas3_gemm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDAUVMSPACE.cpp.o -MF CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/gemm/Blas3_gemm_eti_DOUBLE_LAYOUTL
            EFT_EXECSPACE_CUDA_MEMSPACE_CUDAUVMSPACE.cpp.o.d -o CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/gemm/Blas3_gemm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDAUVMSPACE.cpp.o -c /wclustre/accelsim/sajid
            /packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src/impl/generated_specializations_cpp/gemm/Blas3_gemm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPA
            CE_CUDAUVMSPACE.cpp
     300    [ 41%] Building CXX object src/CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/gemm/Blas3_gemm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cpp.o
     301    cd /wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src && /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientif
            ic7-cascadelake/gcc-9.4.0/kokkos-nvcc-wrapper-3.2.00-geqoj5bmyrvej2s4tefwzvd7thcjk4zj/bin/nvcc_wrapper -DKOKKOS_DEPENDENCE -Dkokkoskernels_EXPORTS -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-
            kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src
             -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-s
            tage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spa
            ck-src/src/impl/tpls -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/blas -I/wclustre/accelsim/sajid/packages/spack/var/spack/
            stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/blas/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv
            7dz42lvog2t/spack-src/src/sparse -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/sparse/impl -I/wclustre/accelsim/sajid/packag
            es/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/graph -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawl
            qf43snj4qgc36lv7dz42lvog2t/spack-src/src/graph/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched -I/wclustre/accels
            im/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched/dense -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokko
            s-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched/dense/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src
            /src/batched/sparse -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched/sparse/impl -I/wclustre/accelsim/sajid/packages/sp
            ack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/common -isystem /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-9.4.0/kokkos-3.5
            .00-fk5fpcsh7o3ih7bkd3o6xvsikjr27mq3/include -O2 -g -DNDEBUG -fPIC -march=skylake-avx512 -mtune=skylake-avx512 -mrtm -expt-extended-lambda -Wext-lambda-captures-this -arch=sm_70 -std=c++14 -MD -MT src/CMakeFiles/kokkoskernel
            s.dir/impl/generated_specializations_cpp/gemm/Blas3_gemm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cpp.o -MF CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/gemm/Blas3_gemm_eti_DOUBLE_LAYOUTLE
            FT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cpp.o.d -o CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/gemm/Blas3_gemm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPACE_HOSTSPACE.cpp.o -c /wclustre/accelsim/sajid/pa
            ckages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src/impl/generated_specializations_cpp/gemm/Blas3_gemm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_SERIAL_MEMSPAC
            E_HOSTSPACE.cpp
     302    [ 41%] Building CXX object src/CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/trsm/Blas3_trsm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o
     303    cd /wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src && /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientif
            ic7-cascadelake/gcc-9.4.0/kokkos-nvcc-wrapper-3.2.00-geqoj5bmyrvej2s4tefwzvd7thcjk4zj/bin/nvcc_wrapper -DKOKKOS_DEPENDENCE -Dkokkoskernels_EXPORTS -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-
            kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src
             -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-s
            tage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spa
            ck-src/src/impl/tpls -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/blas -I/wclustre/accelsim/sajid/packages/spack/var/spack/
            stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/blas/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv
            7dz42lvog2t/spack-src/src/sparse -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/sparse/impl -I/wclustre/accelsim/sajid/packag
            es/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/graph -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawl
            qf43snj4qgc36lv7dz42lvog2t/spack-src/src/graph/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched -I/wclustre/accels
            im/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched/dense -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokko
            s-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched/dense/impl -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src
            /src/batched/sparse -I/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/batched/sparse/impl -I/wclustre/accelsim/sajid/packages/sp
            ack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-src/src/common -isystem /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-cascadelake/gcc-9.4.0/kokkos-3.5
            .00-fk5fpcsh7o3ih7bkd3o6xvsikjr27mq3/include -O2 -g -DNDEBUG -fPIC -march=skylake-avx512 -mtune=skylake-avx512 -mrtm -expt-extended-lambda -Wext-lambda-captures-this -arch=sm_70 -std=c++14 -MD -MT src/CMakeFiles/kokkoskernel
            s.dir/impl/generated_specializations_cpp/trsm/Blas3_trsm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o -MF CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/trsm/Blas3_trsm_eti_DOUBLE_LAYOUTLEFT
            _EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o.d -o CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/trsm/Blas3_trsm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o -c /wclustre/accelsim/sajid/packages
            /spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src/impl/generated_specializations_cpp/trsm/Blas3_trsm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASP
            ACE.cpp
  >> 304    /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-ivybridge/gcc-4.8.5/gcc-9.4.0-2476yvebsimou3rqk4mbehqolhlj56hn/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512fintrin.h(9146): error: identifier "__builtin
            _ia32_rndscaless_round" is undefined
     305
  >> 306    /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-ivybridge/gcc-4.8.5/gcc-9.4.0-2476yvebsimou3rqk4mbehqolhlj56hn/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512fintrin.h(9155): error: identifier "__builtin
            _ia32_rndscalesd_round" is undefined
     307
  >> 308    /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-ivybridge/gcc-4.8.5/gcc-9.4.0-2476yvebsimou3rqk4mbehqolhlj56hn/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512fintrin.h(14797): error: identifier "__builti
            n_ia32_rndscaless_round" is undefined
     309
  >> 310    /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-ivybridge/gcc-4.8.5/gcc-9.4.0-2476yvebsimou3rqk4mbehqolhlj56hn/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512fintrin.h(14806): error: identifier "__builti
            n_ia32_rndscalesd_round" is undefined
     311
  >> 312    /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-ivybridge/gcc-4.8.5/gcc-9.4.0-2476yvebsimou3rqk4mbehqolhlj56hn/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512dqintrin.h(1365): error: identifier "__builti
            n_ia32_fpclassss" is undefined
     313
  >> 314    /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-ivybridge/gcc-4.8.5/gcc-9.4.0-2476yvebsimou3rqk4mbehqolhlj56hn/lib/gcc/x86_64-pc-linux-gnu/9.4.0/include/avx512dqintrin.h(1372): error: identifier "__builti
            n_ia32_fpclasssd" is undefined
     315
     316    6 errors detected in the compilation of "/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq/src/impl/generated_specialization
            s_cpp/trsm/Blas3_trsm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp".
  >> 317    make[2]: *** [src/CMakeFiles/kokkoskernels.dir/impl/generated_specializations_cpp/trsm/Blas3_trsm_eti_DOUBLE_LAYOUTLEFT_EXECSPACE_CUDA_MEMSPACE_CUDASPACE.cpp.o] Error 2
     318    make[2]: Leaving directory `/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq'
  >> 319    make[1]: *** [src/CMakeFiles/kokkoskernels.dir/all] Error 2
     320    make[1]: Leaving directory `/wclustre/accelsim/sajid/packages/spack/var/spack/stage/sasyed/spack-stage-kokkos-kernels-3.5.00-6qlawlqf43snj4qgc36lv7dz42lvog2t/spack-build-6qlawlq'
  >> 321    make: *** [all] Error 2

Spec for kokkos-kernels (built as part of a spack-environment) is : 

==> Concretized kokkos-kernels@3.5.00+blas+cuda cuda_arch=70
 -   6qlawlq  kokkos-kernels@3.5.00%gcc@9.4.0+blas~cblas~cublas+cuda~cusparse~ipo~lapack~lapacke~mkl~openmp~pthread~serial+shared~superlu build_type=RelWithDebInfo cuda_arch=70 execspace_cuda=auto execspace_openmp=auto execspace_serial=auto execspace_threads=auto layouts=left memspace_cudaspace=auto memspace_cudauvmspace=auto offsets=int,size_t ordinals=int scalars=double arch=linux-scientific7-cascadelake
[+]  564snas      ^cmake@3.22.1%gcc@9.4.0~doc+ncurses+openssl+ownlibs~qt build_type=Release arch=linux-scientific7-cascadelake
[+]  3poi7ok          ^ncurses@6.2%gcc@9.4.0~symlinks+termlib abi=none arch=linux-scientific7-cascadelake
[+]  jz4ovym              ^pkgconf@1.8.0%gcc@9.4.0 arch=linux-scientific7-cascadelake
[+]  ehmtnj4          ^openssl@1.1.1m%gcc@9.4.0~docs certs=system arch=linux-scientific7-cascadelake
[+]  3ujga7x              ^perl@5.34.0%gcc@9.4.0+cpanm+shared+threads arch=linux-scientific7-cascadelake
[+]  6l7oukz                  ^berkeley-db@18.1.40%gcc@9.4.0+cxx~docs+stl patches=b231fcc4d5cff05e5c3a4814f6a5af0e9a966428dc2176540d2c05aff41de522 arch=linux-scientific7-cascadelake
[+]  bfce7dn                  ^bzip2@1.0.8%gcc@9.4.0~debug~pic+shared arch=linux-scientific7-cascadelake
[+]  bt6qcc7                      ^diffutils@3.8%gcc@9.4.0 arch=linux-scientific7-cascadelake
[+]  rgkyxly                          ^libiconv@1.16%gcc@9.4.0 libs=shared,static arch=linux-scientific7-cascadelake
[+]  xkendok                  ^gdbm@1.19%gcc@9.4.0 arch=linux-scientific7-cascadelake
[+]  2bzfa47                      ^readline@8.1%gcc@9.4.0 arch=linux-scientific7-cascadelake
[+]  ktzgdrr                  ^zlib@1.2.11%gcc@9.4.0+optimize+pic+shared arch=linux-scientific7-cascadelake
[+]  jvgv64b      ^cuda@11.4.100%gcc@9.4.0~dev arch=linux-scientific7-cascadelake
[+]  fk5fpcs      ^kokkos@3.5.00%gcc@9.4.0~aggressive_vectorization~compiler_warnings+cuda~cuda_constexpr+cuda_lambda~cuda_ldg_intrinsic~cuda_relocatable_device_code~cuda_uvm~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples~explicit_instantiation~hpx~hpx_async_dispatch~hwloc~ipo~memkind~numactl~openmp~pic+profiling~profiling_load_print~pthread~qthread~rocm+serial+shared~sycl~tests~tuning+wrapper build_type=RelWithDebInfo cuda_arch=70 std=14 arch=linux-scientific7-cascadelake
[+]  geqoj5b          ^kokkos-nvcc-wrapper@3.2.00%gcc@9.4.0+mpi arch=linux-scientific7-cascadelake
[+]  nxisrhn              ^openmpi@4.1.2%gcc@9.4.0~atomics+cuda~cxx~cxx_exceptions+gpfs~internal-hwloc~java~legacylaunchers~lustre~memchecker~pmi~pmix+romio~singularity~sqlite3+static~thread_multiple+vt+wrapper-rpath fabrics=auto schedulers=none arch=linux-scientific7-cascadelake
[+]  fyzivp2                  ^hwloc@2.7.0%gcc@9.4.0~cairo+cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-scientific7-cascadelake
[+]  5e4mpx7                      ^libpciaccess@0.16%gcc@9.4.0 arch=linux-scientific7-cascadelake
[+]  6w6tcp3                          ^libtool@2.4.6%gcc@9.4.0 arch=linux-scientific7-cascadelake
[+]  jc2xq74                              ^m4@1.4.19%gcc@9.4.0+sigsegv patches=9dc5fbd0d5cb1037ab1e6d0ecc74a30df218d0a94bdd5a02759a97f62daca573,bfdffa7c2eb01021d5849b36972c069693654ad826c1a20b53534009a4ec7a89 arch=linux-scientific7-cascadelake
[+]  jxb6ru2                                  ^libsigsegv@2.13%gcc@9.4.0 arch=linux-scientific7-cascadelake
[+]  5owalpo                          ^util-macros@1.19.3%gcc@9.4.0 arch=linux-scientific7-cascadelake
[+]  5z4ugky                      ^libxml2@2.9.12%gcc@9.4.0~python arch=linux-scientific7-cascadelake
[+]  yktgy54                          ^xz@5.2.5%gcc@9.4.0~pic libs=shared,static arch=linux-scientific7-cascadelake
[+]  gbmm62k                  ^libevent@2.1.12%gcc@9.4.0+openssl arch=linux-scientific7-cascadelake
[+]  aymoqm5                  ^numactl@2.0.14%gcc@9.4.0 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-scientific7-cascadelake
[+]  gxyo6ws                      ^autoconf@2.69%gcc@9.4.0 patches=35c449281546376449766f92d49fc121ca50e330e60fefcfc9be2af3253082c2,7793209b33013dc0f81208718c68440c5aae80e7a1c4b8d336e382525af791a7,a49dd5bac3b62daa0ff688ab4d508d71dbd2f4f8d7e2a02321926346161bf3ee arch=linux-scientific7-cascadelake
[+]  oq5sxd6                      ^automake@1.16.5%gcc@9.4.0 arch=linux-scientific7-cascadelake
[+]  bjs3c6d                  ^openssh@8.8p1%gcc@9.4.0 arch=linux-scientific7-cascadelake
[+]  7amxji4                      ^libedit@3.1-20210216%gcc@9.4.0 arch=linux-scientific7-cascadelake
[+]  a72c5p2      ^openblas@0.3.19%gcc@9.4.0~bignuma~consistent_fpcsr~ilp64+locking+pic+shared symbol_suffix=none threads=openmp arch=linux-scientific7-cascadelake

spack logs:

spack-build-env-mods.txt spack-build-out.txt spack-configure-args.txt spack-build-01-cmake-out.txt spack-build-02-build-out.txt spack-build-env.txt

lucbv commented 2 years ago

@s-sajid-ali We will try to reproduce this and let you know. It would help to know how Kokkos was configured as Kokkos Kernels uses some of the variables defined in Kokkos' installation.

s-sajid-ali commented 2 years ago

Here are the build logs for kokkos with the spec: kokkos@3.5.00%gcc@9.4.0~aggressive_vectorization~compiler_warnings+cuda~cuda_constexpr+cuda_lambda~cuda_ldg_intrinsic~cuda_relocatable_device_code~cuda_uvm~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples~explicit_instantiation~hpx~hpx_async_dispatch~hwloc~ipo~memkind~numactl~openmp~pic+profiling~profiling_load_print~pthread~qthread~rocm+serial+shared~sycl~tests~tuning+wrapper build_type=RelWithDebInfo cuda_arch=70 std=14 arch=linux-scientific7-cascadelake

spack-configure-args.txt spack-build-01-cmake-out.txt spack-build-02-build-out.txt spack-build-03-install-out.txt spack-build-env.txt spack-build-out.txt

Let me know if anything else is needed. Thanks!

JBludau commented 2 years ago

@lucbv is this still under investigation or did it stall?

mfaysal101 commented 1 year ago

@lucbv I am experiencing the same issue just like @s-sajid-ali but without using spack. Will it help if I post the steps that lead to me this?