Closed G-071 closed 5 months ago
I get a build error in asio
with cxxstd=20
:
In file included from /tmp/junghans/spack-stage/spack-stage-asio-1.21.0-rglwk2e2se4iheqn3wwinm4rg44ekkdg/spack-src/asio/src/examples/cpp17/../../../include/asio/co_spawn.hpp:22,
from coroutines_ts/echo_server.cpp:11:
/tmp/junghans/spack-stage/spack-stage-asio-1.21.0-rglwk2e2se4iheqn3wwinm4rg44ekkdg/spack-src/asio/src/examples/cpp17/../../../include/asio/awaitable.hpp: In constructor 'asio::awaitable<T, Executor>::awaitable(asio::awaitable<T, Executor>&&)':
/tmp/junghans/spack-stage/spack-stage-asio-1.21.0-rglwk2e2se4iheqn3wwinm4rg44ekkdg/spack-src/asio/src/examples/cpp17/../../../include/asio/awaitable.hpp:67:19: error: 'exchange' is not a member of 'std'; did you mean 'std::__atomic_impl::exchange'?
67 | : frame_(std::exchange(other.frame_, nullptr))
| ^~~~~~~~
Hmm, what might work here is keeping HPX (and thus asio) at C++17 even though we use 20 for Octo-Tiger. I just reverted the last commit that propagated the cxxflag automatically: it should now be possible to just use cxxstd=20
for Octo-Tiger.
[ 3%] Building CUDA object CMakeFiles/hydrolib.dir/src/unitiger/hydro_impl/hydro_kernel_interface.cpp.o
/vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/kokkos-4.0.01-hswovegdbcq2qg7dbzptx3yp6whqt4pn/bin/kokkos_launch_compiler /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/kokkos-4.0.01-hswovegdbcq2qg7dbzptx3yp6whqt4pn/bin/nvcc_wrapper /vast/home/junghans/octotiger-spack/spack/lib/spack/env/gcc/g++ /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/cuda-12.3.0-waox3sh3wy5zfvo46wvnoqmfluh24fbe/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-11.4.1/gcc-12.3.0-5oi3muh65s6a3efcuqmnyti3lhganawe/bin/g++ -DASIO_DISABLE_CONSTEXPR -DCPPUDDLE_HAVE_HPX -DCPPUDDLE_HAVE_HPX_AWARE_ALLOCATORS -DCPPUDDLE_HAVE_HPX_MUTEX -DCPPUDDLE_HAVE_MAX_NUMBER_GPUS=1 -DCPPUDDLE_HAVE_NUMBER_BUCKETS=128 -DH5_BUILT_AS_DYNAMIC_LIB -DHPX_KOKKOS_CUDA_FUTURE_TYPE=0 -DHPX_KOKKOS_SYCL_FUTURE_TYPE=0 -DHPX_LIBRARY_EXPORTS -DHPX_WITH_CUDA -DOCTOTIGER_GRIDDIM=8 -DOCTOTIGER_HAVE_CUDA -DOCTOTIGER_HAVE_KOKKOS -DOCTOTIGER_HAVE_STD_EXPERIMENTAL_SIMD -DOCTOTIGER_HAVE_VC -DOCTOTIGER_KOKKOS_HYDRO_TASKS=1 -DOCTOTIGER_KOKKOS_MONOPOLE_TASKS=1 -DOCTOTIGER_KOKKOS_MULTIPOLE_TASKS=1 -DOCTOTIGER_KOKKOS_SIMD_SVE -DOCTOTIGER_MAX_NUMBER_FIELDS=15 -DOCTOTIGER_THETA_MINIMUM=0.34 -DSVE_LEN=256 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -D_POSIX_C_SOURCE=200809L -Dhydrolib_EXPORTS --options-file CMakeFiles/hydrolib.dir/includes_CUDA.rsp -O3 -DNDEBUG -std=c++20 --generate-code=arch=compute_90,code=[compute_90,sm_90] -Xcompiler=-fPIC -msve-vector-bits=256 -w --extended-lambda --default-stream per-thread --expt-relaxed-constexpr --ftz=true --prec-div=false --prec-sqrt=false --fmad=false --expt-extended-lambda -expt-relaxed-constexpr -Xptxas=-v -Xcompiler=-march=native,-ffp-contract=off -MD -MT CMakeFiles/hydrolib.dir/src/unitiger/hydro_impl/hydro_kernel_interface.cpp.o -MF CMakeFiles/hydrolib.dir/src/unitiger/hydro_impl/hydro_kernel_interface.cpp.o.d -x cu -c /vast/home/junghans/octotiger-spack/octotiger/src/unitiger/hydro_impl/hydro_kernel_interface.cpp -o CMakeFiles/hydrolib.dir/src/unitiger/hydro_impl/hydro_kernel_interface.cpp.o
/vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-1.74.0-3o6b54jftc65y5bwmi7vcshobciz4syy/include/boost/iostreams/detail/buffer.hpp(43): error: class "std::allocator<char>" has no member "rebind"
typedef typename Alloc::template rebind<Ch>::other allocator_type;
^
detected during:
instantiation of class "boost::iostreams::detail::basic_buffer<Ch, Alloc> [with Ch=char, Alloc=std::allocator<char>]" at line 52 of /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-1.74.0-3o6b54jftc65y5bwmi7vcshobciz4syy/include/boost/iostreams/detail/double_object.hpp
instantiation of class "boost::iostreams::detail::single_object_holder<T> [with T=boost::iostreams::detail::basic_buffer<char, std::allocator<char>>]" at line 84 of /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-1.74.0-3o6b54jftc65y5bwmi7vcshobciz4syy/include/boost/iostreams/detail/double_object.hpp
instantiation of class "boost::iostreams::detail::double_object<T, IsDouble> [with T=boost::iostreams::detail::basic_buffer<char, std::allocator<char>>, IsDouble=boost::is_convertible<boost::iostreams::output, boost::iostreams::detail::two_sequence>]" at line 138 of /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-1.74.0-3o6b54jftc65y5bwmi7vcshobciz4syy/include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp
instantiation of class "boost::iostreams::detail::indirect_streambuf<T, Tr, Alloc, Mode> [with T=hpx::iostreams::detail::buffer_sink<char>, Tr=std::char_traits<char>, Alloc=std::allocator<char>, Mode=boost::iostreams::output]" at line 65 of /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-1.74.0-3o6b54jftc65y5bwmi7vcshobciz4syy/include/boost/iostreams/stream_buffer.hpp
instantiation of class "boost::iostreams::stream_buffer<T, Tr, Alloc, Mode> [with T=hpx::iostreams::detail::buffer_sink<char>, Tr=std::char_traits<char>, Alloc=std::allocator<char>, Mode=boost::iostreams::output]" at line 127 of /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-1.74.0-3o6b54jftc65y5bwmi7vcshobciz4syy/include/boost/utility/base_from_member.hpp
instantiation of class "boost::base_from_member<MemberType, UniqueID> [with MemberType=boost::iostreams::stream_buffer<hpx::iostreams::detail::buffer_sink<char>, std::char_traits<char>, std::allocator<char>, boost::iostreams::output>, UniqueID=0]" at line 81 of /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-1.74.0-3o6b54jftc65y5bwmi7vcshobciz4syy/include/boost/iostreams/stream.hpp
instantiation of class "boost::iostreams::detail::stream_base<Device, Tr, Alloc, Base> [with Device=hpx::iostreams::detail::buffer_sink<char>, Tr=std::char_traits<char>, Alloc=std::allocator<char>, Base=std::basic_ostream<char, std::char_traits<char>>]" at line 128 of /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-1.74.0-3o6b54jftc65y5bwmi7vcshobciz4syy/include/boost/iostreams/stream.hpp
instantiation of class "boost::iostreams::stream<Device, Tr, Alloc> [with Device=hpx::iostreams::detail::buffer_sink<char>, Tr=std::char_traits<char>, Alloc=std::allocator<char>]" at line 166 of /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/hpx-1.9.1-xtda4cnlwsa2tpcyps4t7ngbh7s2osd7/include/hpx/components/iostreams/ostream.hpp
instantiation of class "hpx::iostreams::ostream<Char, Sink> [with Char=char, Sink=hpx::iostreams::detail::buffer_sink<char>]" at line 99 of /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/cppuddle-0.3.1-spcyacohuk3qr3sjfr3phdnxqkkqlkwt/include/aggregation_manager.hpp
I think I saw that before, the problem is that boost needs to be cxx=20.
Let me try to bump asio, v1.21.0 seems a bit old.
Ok, asio@1.22.2
works for me.
version("1.22.2", sha256="985fc2d522f32d232d8386d2fa4ac6f2b25a0cad30495bf2e2e7997bce743f0b")
Now it fails in octotiger again:
/vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-1.74.0-3o6b54jftc65y5bwmi7vcshobciz4syy/include/boost/iostreams/detail/buffer.hpp:43:56: error: no class template named 'rebind' in 'class std::allocator<char>'
43 | typedef typename Alloc::template rebind<Ch>::other allocator_type;
| ^~~~~~~~~~~~~~
In file included from /vast/home/junghans/octotiger-spack/octotiger/octotiger/common_kernel/kokkos_simd.hpp:32,
from /vast/home/junghans/octotiger-spack/octotiger/octotiger/unitiger/hydro_impl/flux_kernel_templates.hpp:11,
from /vast/home/junghans/octotiger-spack/octotiger/src/unitiger/hydro_impl/hydro_cuda_interface.cpp:21:
/vast/home/junghans/octotiger-spack/octotiger/octotiger/common_kernel/std_simd.hpp:51:28: error: 'sve' has not been declared
51 | namespace SIMD_NAMESPACE = sve::experimental::parallelism_v2;
| ^~~
with
Input spec
--------------------------------
octotiger@develop%gcc@12.3.0+boost_multiprecision+cuda+kokkos cuda_arch=90 cxxstd=20 simd_extension=SVE simd_library=STD sve_length=256
^cuda@12.3.0
^hpx+generic_coroutines max_cpu_count=128
Concretized
--------------------------------
octotiger@develop%gcc@12.3.0+boost_multiprecision+cuda~fast_fp_contract~ipo+kokkos~kokkos_hpx_kernels~rocm~sycl build_system=cmake build_type=Release cuda_arch=90 cxxstd=20 generator=make griddim=8 hydro_host_tasks=1 monopole_host_tasks=1 multipole_host_tasks=1 simd_extension=SVE simd_library=STD sve_length=256 theta_minimum=0.34 arch=linux-rhel9-neoverse_n1
^boost@1.82.0%gcc@12.3.0+atomic+chrono~clanglibcpp~container+context~contract~coroutine+date_time~debug+exception~fiber+filesystem+graph~graph_parallel~icu+iostreams~json+locale+log+math~mpi+multithreaded~nowide~numpy~pic+program_options~python+random+regex+serialization+shared+signals~singlethreaded~stacktrace+system~taggedlayout+test+thread+timer~type_erasure~versionedlayout+wave build_system=generic context-impl=fcontext cxxstd=2a patches=a440f96,a7c807f visibility=hidden arch=linux-rhel9-neoverse_n1
^bzip2@1.0.8%gcc@12.3.0~debug~pic+shared build_system=generic arch=linux-rhel9-neoverse_n1
^diffutils@3.9%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^xz@5.4.1%gcc@12.3.0~pic build_system=autotools libs=shared,static arch=linux-rhel9-neoverse_n1
^zlib@1.2.13%gcc@12.3.0+optimize+pic+shared build_system=makefile arch=linux-rhel9-neoverse_n1
^zstd@1.5.5%gcc@12.3.0+programs build_system=makefile compression=none libs=shared,static arch=linux-rhel9-neoverse_n1
^cmake@3.26.3%gcc@12.3.0~doc+ncurses+ownlibs~qt build_system=generic build_type=Release arch=linux-rhel9-neoverse_n1
^ncurses@6.4%gcc@12.3.0~symlinks+termlib abi=none build_system=autotools arch=linux-rhel9-neoverse_n1
^openssl@1.1.1t%gcc@12.3.0~docs~shared build_system=generic certs=mozilla arch=linux-rhel9-neoverse_n1
^ca-certificates-mozilla@2023-01-10%gcc@12.3.0 build_system=generic arch=linux-rhel9-neoverse_n1
^perl@5.36.0%gcc@12.3.0+cpanm+open+shared+threads build_system=generic arch=linux-rhel9-neoverse_n1
^berkeley-db@18.1.40%gcc@12.3.0+cxx~docs+stl build_system=autotools patches=26090f4,b231fcc arch=linux-rhel9-neoverse_n1
^cppuddle@0.3.1%gcc@12.3.0~allocator_counters+buffer_content_recycling+buffer_recycling~enable_gpu_tests+executor_recycling+hpx~ipo build_system=cmake build_type=Release generator=make max_number_gpus=1 number_buffer_buckets=128 arch=linux-rhel9-neoverse_n1
^cuda@12.3.0%gcc@12.3.0~allow-unsupported-compilers~dev build_system=generic arch=linux-rhel9-neoverse_n1
^libxml2@2.10.3%gcc@12.3.0~python build_system=autotools arch=linux-rhel9-neoverse_n1
^libiconv@1.17%gcc@12.3.0 build_system=autotools libs=shared,static arch=linux-rhel9-neoverse_n1
^gmake@4.4.1%gcc@12.3.0~guile build_system=autotools arch=linux-rhel9-neoverse_n1
^gnuconfig@2022-09-17%gcc@12.3.0 build_system=generic arch=linux-rhel9-neoverse_n1
^hdf5@1.14.1-2%gcc@12.3.0~cxx~fortran+hl~ipo~java~map~mpi+shared+szip+threadsafe+tools api=default build_system=cmake build_type=Release generator=make arch=linux-rhel9-neoverse_n1
^libaec@1.0.6%gcc@12.3.0~ipo+shared build_system=cmake build_type=Release generator=make arch=linux-rhel9-neoverse_n1
^pkgconf@1.9.5%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^hpx@1.9.1%gcc@12.3.0+async_cuda+async_gpu_futures~async_mpi+cuda~examples+generic_coroutines~ipo~lci_pp_log~lci_pp_pcounter~rocm~sycl~tools build_system=cmake build_type=Release cuda_arch=90 cxxstd=20 generator=ninja instrumentation=none malloc=tcmalloc max_cpu_count=128 networking=tcp sycl_target_arch=none arch=linux-rhel9-neoverse_n1
^asio@1.22.2%gcc@12.3.0~boost_coroutine~boost_regex~separate_compilation build_system=autotools cxxstd=2a arch=linux-rhel9-neoverse_n1
^git@2.40.0%gcc@12.3.0+man+nls+perl+subtree~svn~tcltk build_system=autotools arch=linux-rhel9-neoverse_n1
^curl@8.0.1%gcc@12.3.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2~nghttp2 build_system=autotools libs=shared,static tls=openssl arch=linux-rhel9-neoverse_n1
^expat@2.5.0%gcc@12.3.0+libbsd build_system=autotools arch=linux-rhel9-neoverse_n1
^libbsd@0.11.7%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^libmd@1.0.4%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^gettext@0.21.1%gcc@12.3.0+bzip2+curses+git~libunistring+libxml2+tar+xz build_system=autotools arch=linux-rhel9-neoverse_n1
^tar@1.34%gcc@12.3.0 build_system=autotools zip=pigz arch=linux-rhel9-neoverse_n1
^pigz@2.7%gcc@12.3.0 build_system=makefile arch=linux-rhel9-neoverse_n1
^libidn2@2.3.4%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^libunistring@1.1%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^openssh@9.3p1%gcc@12.3.0+gssapi build_system=autotools arch=linux-rhel9-neoverse_n1
^krb5@1.20.1%gcc@12.3.0+shared build_system=autotools arch=linux-rhel9-neoverse_n1
^bison@3.8.2%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^libedit@3.1-20210216%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^pcre2@10.42%gcc@12.3.0~jit+multibyte build_system=autotools arch=linux-rhel9-neoverse_n1
^gperftools@2.10%gcc@12.3.0+debugalloc~dynamic_sized_delete_support+libunwind~sized_delete build_system=autotools arch=linux-rhel9-neoverse_n1
^libunwind@1.6.2%gcc@12.3.0~block_signals~conservative_checks~cxx_exceptions~debug~debug_frame+docs~pic+tests+weak_backtrace~xz~zlib build_system=autotools components=none libs=shared,static arch=linux-rhel9-neoverse_n1
^hwloc@2.9.1%gcc@12.3.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml~oneapi-level-zero~opencl+pci~rocm build_system=autotools libs=shared,static arch=linux-rhel9-neoverse_n1
^libpciaccess@0.17%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^util-macros@1.19.3%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^ninja@1.11.1%gcc@12.3.0+re2c build_system=generic arch=linux-rhel9-neoverse_n1
^re2c@2.2%gcc@12.3.0 build_system=generic arch=linux-rhel9-neoverse_n1
^python@3.10.10%gcc@12.3.0+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=0d98e93,7d40923,f2fd060 arch=linux-rhel9-neoverse_n1
^gdbm@1.23%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^libffi@3.4.4%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^libxcrypt@4.4.33%gcc@12.3.0~obsolete_api build_system=autotools arch=linux-rhel9-neoverse_n1
^sqlite@3.40.1%gcc@12.3.0+column_metadata+dynamic_extensions+fts~functions+rtree build_system=autotools arch=linux-rhel9-neoverse_n1
^util-linux-uuid@2.38.1%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^hpx-kokkos@0.4.0%gcc@12.3.0+cuda~ipo~rocm~sycl build_system=cmake build_type=Release cuda_arch=90 cxxstd=20 future_type=polling generator=make arch=linux-rhel9-neoverse_n1
^kokkos@4.0.01%gcc@12.3.0+aggressive_vectorization~compiler_warnings+cuda+cuda_constexpr+cuda_lambda~cuda_ldg_intrinsic~cuda_relocatable_device_code~cuda_uvm~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples+hpx+hpx_async_dispatch~hwloc~ipo~memkind~numactl~openmp~openmptarget~pic~rocm+serial+shared~sycl~tests~threads~tuning+wrapper build_system=cmake build_type=Release cuda_arch=90 generator=make intel_gpu_arch=none patches=b26a011,b475d96 std=20 arch=linux-rhel9-neoverse_n1
^kokkos-nvcc-wrapper@4.0.01%gcc@12.3.0 build_system=generic patches=b475d96 arch=linux-rhel9-neoverse_n1
^silo@4.11%gcc@12.3.0+fortran+fpzip+hdf5+hzip+mpi+pic+shared~silex build_system=autotools patches=451c4c5,a081263,eb2a3a0,fa050e0 arch=linux-rhel9-neoverse_n1
^autoconf@2.69%gcc@12.3.0 build_system=autotools patches=35c4492,7793209,a49dd5b arch=linux-rhel9-neoverse_n1
^autoconf-archive@2023.02.20%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^automake@1.16.5%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^libtool@2.4.7%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^m4@1.4.19%gcc@12.3.0+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-rhel9-neoverse_n1
^libsigsegv@2.14%gcc@12.3.0 build_system=autotools arch=linux-rhel9-neoverse_n1
^openmpi@4.1.5%gcc@12.3.0~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java~legacylaunchers~lustre~memchecker~orterunprefix+romio+rsh~singularity+static+vt+wrapper-rpath build_system=autotools fabrics=none schedulers=none arch=linux-rhel9-neoverse_n1
^numactl@2.0.14%gcc@12.3.0 build_system=autotools patches=4e1d78c,62fc8a8,ff37630 arch=linux-rhel9-neoverse_n1
^pmix@4.2.3%gcc@12.3.0~docs+pmi_backwards_compatibility~python~restful build_system=autotools arch=linux-rhel9-neoverse_n1
^libevent@2.1.12%gcc@12.3.0+openssl build_system=autotools arch=linux-rhel9-neoverse_n1
^readline@8.2%gcc@12.3.0 build_system=autotools patches=bbf97f1 arch=linux-rhel9-neoverse_n1
^vc@1.4.1%gcc@12.3.0~ipo build_system=cmake build_type=Release generator=make arch=linux-rhel9-neoverse_n1
I am bit confused why only boost-1.74.0 is pulled in....
Ok, I found that, it picks up boost-1.82
after I did a git clean
to remote the old spack-build
.
But the error persists:
/vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-1.82.0-3vpmlzz4ixs55jc46n62uqvs7mwzkpqr/include/boost/iostreams/detail/buffer.hpp:43:56: error: no class template named 'rebind' in 'class std::allocator<char>'
43 | typedef typename Alloc::template rebind<Ch>::other allocator_type;
| ^~~~~~~~~~~~~~
In file included from /vast/home/junghans/octotiger-spack/octotiger/octotiger/common_kernel/kokkos_simd.hpp:32,
from /vast/home/junghans/octotiger-spack/octotiger/octotiger/unitiger/hydro_impl/flux_kernel_templates.hpp:11,
from /vast/home/junghans/octotiger-spack/octotiger/src/unitiger/hydro_impl/hydro_cuda_interface.cpp:21:
/vast/home/junghans/octotiger-spack/octotiger/octotiger/common_kernel/std_simd.hpp:51:28: error: 'sve' has not been declared
51 | namespace SIMD_NAMESPACE = sve::experimental::parallelism_v2;
| ^~~
The problem persists even with boost-1.84.0
Is there any more information in the build log on the first error (rebind)? Is it still coming from aggregation_manager.hpp:98 / hpx::cout? If that's the case, I might be able to simply remove that part of the header for SVE builds without us losing any significant features.
As for the second error: That one is weird! The compiler appears to be able to include the SVE header without any issues, but can't find the SVE namespace defined in it. Glancing at the SVE header itself, the only potential cause for this I can see is if __ARM_FEATURE_SVE is not defined for some reason (so it might be some issue in our build system).
To keep things simple for now: Could you try creating an SVE build without CUDA enabled? (basically the same build, just with ~cuda and without cuda_arch). Mixing SVE and CUDA builds is just not something we have tested before, so I would suggest that we stick with pure SVE for now until we can get that running first.
Yup it is still in aggregation_manager.hpp
. (See will shared the log with you via email)
WIth cuda, tis fails with:
/vast/home/junghans/octotiger-spack/spack/lib/spack/env/gcc/g++ -DCPPUDDLE_HAVE_HPX -DCPPUDDLE_HAVE_HPX_AWARE_ALLOCATORS -DCPPUDDLE_HAVE_HPX_MUTEX -DCPPUDDLE_HAVE_MAX_NUMBER_GPUS=1 -DCPPUDDLE_HAVE_NUMBER_
BUCKETS=128 -DH5_BUILT_AS_DYNAMIC_LIB -DHPX_KOKKOS_CUDA_FUTURE_TYPE=0 -DHPX_KOKKOS_SYCL_FUTURE_TYPE=0 -DHPX_LIBRARY_EXPORTS -DHPX_WITH_CUDA -DKOKKOS_DEPENDENCE -DOCTOTIGER_GRIDDIM=8 -DOCTOTIGER_HAVE_KOKKO
S -DOCTOTIGER_HAVE_STD_EXPERIMENTAL_SIMD -DOCTOTIGER_HAVE_VC -DOCTOTIGER_KOKKOS_HYDRO_TASKS=1 -DOCTOTIGER_KOKKOS_MONOPOLE_TASKS=1 -DOCTOTIGER_KOKKOS_MULTIPOLE_TASKS=1 -DOCTOTIGER_KOKKOS_SIMD_SVE -DOCTOTIG
ER_MAX_NUMBER_FIELDS=15 -DOCTOTIGER_THETA_MINIMUM=0.34 -DSVE_LEN=256 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -D_POSIX_C_SOURCE=200809L -Dhydrolib_EXPORTS -I/vast/hom
e/junghans/octotiger-spack/octotiger -I/vast/home/junghans/octotiger-spack/octotiger/spack-build/_deps/svesimd-src/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse
_n1/gcc-12.3.0/hpx-1.9.1-6ccd5erkj3dgdcbj34oe6y3g7tvpmag4/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-1.84.0-gtc6kjmh2hw3et27lfofdepq5y4ty
2ud/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/hwloc-2.9.1-nsillyjc2h5xhv3k6hs7jvvs6tnivd5d/include -isystem /vast/home/junghans/octotiger-spac
k/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/asio-1.22.2-r4jbde74qjc3c3dq26i7xulb7nkd6dtm/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/si
lo-4.11-f6niwvryvee6jtkldolycxme56ymqgji/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/hdf5-1.14.1-2-i2rtxbxarf54wbpd5xsurtkd7l3mvuvh/include -isy
stem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/zlib-1.2.13-xyz7izlqcwg44f4nkz55apfqzgrlsmo5/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spac
k/linux-rhel9-neoverse_n1/gcc-12.3.0/cppuddle-0.3.1-m3c5cwxelrbwfpwnnpzxbc35geol4snk/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/vc-1.4.1-3sxlec
hnpz3743ldgoqbektzx4zu7nwf/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/kokkos-4.1.00-qdgwacoe5gbkjvcwmlmv2l726gavhvdm/include -isystem /vast/hom
e/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/hpx-kokkos-0.4.0-537vo6kmlgve6d6ltr2xhuljsatprs5p/include -Wno-cpp -fPIC -O3 -DNDEBUG -std=gnu++20 -fPIC -msve-vector-bits=256
-pthread -MD -MT CMakeFiles/hydrolib.dir/src/test_problems/sod/exact_sod.cpp.o -MF CMakeFiles/hydrolib.dir/src/test_problems/sod/exact_sod.cpp.o.d -o CMakeFiles/hydrolib.dir/src/test_problems/sod/exact_s
od.cpp.o -c /vast/home/junghans/octotiger-spack/octotiger/src/test_problems/sod/exact_sod.cpp
In file included from /vast/home/junghans/octotiger-spack/octotiger/octotiger/common_kernel/kokkos_simd.hpp:32,
from /vast/home/junghans/octotiger-spack/octotiger/octotiger/unitiger/hydro_impl/flux_kernel_templates.hpp:11,
from /vast/home/junghans/octotiger-spack/octotiger/octotiger/unitiger/hydro_impl/hydro_kokkos_kernel.hpp:22,
from /vast/home/junghans/octotiger-spack/octotiger/src/unitiger/hydro_impl/hydro_kernel_interface.cpp:16:
/vast/home/junghans/octotiger-spack/octotiger/octotiger/common_kernel/std_simd.hpp:51:28: error: 'sve' has not been declared
51 | namespace SIMD_NAMESPACE = sve::experimental::parallelism_v2;
| ^~~
/vast/home/junghans/octotiger-spack/octotiger/octotiger/common_kernel/std_simd.hpp:51:47: error: 'parallelism_v2' is not a namespace-name
51 | namespace SIMD_NAMESPACE = sve::experimental::parallelism_v2;
| ^~~~~~~~~~~~~~
The PR (and associated Octo-Tiger PR) actually seem to work for me on a Grace preproduction system using and SVE length of 128.
That is with using the octotiger PR (branch name is: allow_different_SVE_lengths
) and the following Spack dev-build:
spack dev-build --until cmake --drop-in bash octotiger@master cxxstd=20 simd_extension=SVE simd_library=STD build_type==RelWithDebInfo sve_length=128 +kokkos +kokkos_hpx_kernels %gcc@12 ^hpx generator=make max_cpu_count=144 +generic_coroutines malloc=mimalloc
networking=lci ^silo~mpi ^lci~vector ^rdma-core~man_pages~pyverbs ^cppuddle@0.3.1 number_buffer_buckets=144
or, to have a more minimal build without the lci parcalport networking part:
spack dev-build --until cmake --drop-in bash octotiger@master cxxstd=20 simd_extension=SVE simd_library=STD build_type==RelWithDebInfo sve_length=128 +kokkos +kokkos_hpx_kernels %gcc@12 ^hpx generator=make max_cpu_count=144 +generic_coroutines malloc=mimalloc ^silo~mpi ^cppuddle@0.3.1 number_buffer_buckets=144
The most annoying part on the system I was on, was actually to get a gcc@12 up and running (I had to disable mpfr and mpc in the gcc Spack recipe to get it to compile without complaining about missing dependencies). Notably, I did not encounter any problems with Boost 1.84 or asio though, so there might be some differences between our setups still.
Some differences I can think of right now:
adapt_for_sycl_on_intel_node
(which has this PR merged in already). This branch uses a newer Kokkos version and a newer Kokkos package recipe which might have helped as well (as Kokkos tweaks the compiler flags for Octo-Tiger -- though I still think it's a bit unlikely that that's the cause of the errors you're encountering).@junghans Could you try a fresh build with the same spack version, spack spec and octo-tiger spack branch?
Here is the concretized spec I was using for the aforementioned, working Spack dev-build:
Input spec
--------------------------------
- octotiger@master%gcc@12+kokkos+kokkos_hpx_kernels build_type==RelWithDebInfo cxxstd=20 simd_extension=SVE simd_library=STD sve_length=128
- ^cppuddle@0.3.1 number_buffer_buckets=144
- ^hpx+generic_coroutines generator=make malloc=mimalloc max_cpu_count=144 networking=lci
- ^lci~vector
- ^rdma-core~man_pages~pyverbs
- ^silo~mpi
Concretized
--------------------------------
- octotiger@master%gcc@12.3.0~boost_multiprecision~cuda~fast_fp_contract~ipo+kokkos+kokkos_hpx_kernels~rocm~sycl build_system=cmake build_type=RelWithDebInfo cxxstd=20 generator=make griddim=8 hydro_host_tasks=1 monopole_host_tasks=1 multipole_host_tasks=1 simd_extension=SVE simd_lib
rary=STD sve_length=128 theta_minimum=0.34 arch=linux-ubuntu22.04-neoverse_v2
[+] ^boost@1.84.0%gcc@12.3.0+atomic+chrono~clanglibcpp~container+context~contract~coroutine+date_time~debug+exception~fiber+filesystem+graph~graph_parallel~icu+iostreams~json+locale+log+math~mpi+multithreaded~nowide~numpy~pic+program_options~python+random+regex+serialization+shared
+signals~singlethreaded~stacktrace+system~taggedlayout+test+thread+timer~type_erasure~versionedlayout+wave build_system=generic context-impl=fcontext cxxstd=17 patches=a440f96 visibility=hidden arch=linux-ubuntu22.04-neoverse_v2
[+] ^bzip2@1.0.8%gcc@12.3.0~debug~pic+shared build_system=generic arch=linux-ubuntu22.04-neoverse_v2
[e] ^diffutils@3.8%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[+] ^xz@5.4.1%gcc@12.3.0~pic build_system=autotools libs=shared,static arch=linux-ubuntu22.04-neoverse_v2
[+] ^zlib-ng@2.1.5%gcc@12.3.0+compat+opt build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[+] ^zstd@1.5.5%gcc@12.3.0~programs build_system=makefile libs=shared,static arch=linux-ubuntu22.04-neoverse_v2
[e] ^cmake@3.22.1%gcc@12.3.0~doc+ncurses+ownlibs build_system=generic build_type=RelWithDebInfo arch=linux-ubuntu22.04-neoverse_v2
[+] ^cppuddle@0.3.1%gcc@12.3.0~allocator_counters+buffer_content_recycling+buffer_recycling~enable_gpu_tests+executor_recycling+hpx~ipo build_system=cmake build_type=RelWithDebInfo generator=make max_number_gpus=1 number_buffer_buckets=144 arch=linux-ubuntu22.04-neoverse_v2
[+] ^gcc-runtime@12.3.0%gcc@12.3.0 build_system=generic arch=linux-ubuntu22.04-neoverse_v2
[e] ^gmake@4.3%gcc@12.3.0~guile build_system=generic patches=599f134 arch=linux-ubuntu22.04-neoverse_v2
[+] ^hdf5@1.14.3%gcc@12.3.0~cxx~fortran+hl~ipo~java~map~mpi+shared+szip+threadsafe+tools api=default build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-ubuntu22.04-neoverse_v2
[+] ^libaec@1.0.6%gcc@12.3.0~ipo+shared build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-ubuntu22.04-neoverse_v2
[e] ^pkgconf@1.4.2%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[+] ^hpx@1.9.1%gcc@12.3.0~async_cuda+async_gpu_futures~async_mpi~cuda~examples+generic_coroutines~ipo~lci_pp_log~lci_pp_pcounter~rocm~sycl~tools build_system=cmake build_type=RelWithDebInfo cxxstd=17 generator=make instrumentation=none malloc=mimalloc max_cpu_count=144 networking=l
ci sycl_target_arch=none arch=linux-ubuntu22.04-neoverse_v2
[+] ^asio@1.28.0%gcc@12.3.0~boost_coroutine~boost_regex~separate_compilation build_system=autotools cxxstd=17 arch=linux-ubuntu22.04-neoverse_v2
[e] ^git@2.39.3%gcc@12.3.0+man+nls+perl+subtree~svn~tcltk build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[+] ^hwloc@2.9.1%gcc@12.3.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml~oneapi-level-zero~opencl+pci~rocm build_system=autotools libs=shared,static arch=linux-ubuntu22.04-neoverse_v2
[+] ^libpciaccess@0.17%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[+] ^util-macros@1.19.3%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[+] ^libxml2@2.10.3%gcc@12.3.0+pic~python+shared build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[+] ^libiconv@1.17%gcc@12.3.0 build_system=autotools libs=shared,static arch=linux-ubuntu22.04-neoverse_v2
[+] ^ncurses@6.4%gcc@12.3.0~symlinks+termlib abi=none build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[+] ^lci@1.7.6%gcc@12.3.0+aligned~benchmarks~debug~debug-slow+default-dreg~docs~examples~gprof+ibv-td~inline-cq~ipo+multithread-progress+native~papi~pcounter+shared~tests~vector build_system=cmake build_type=RelWithDebInfo cache-line=auto completion=am,cq,sync default-max-cqe=a
uto default-max-recvs=auto default-max-sends=auto default-packet-size=auto default-packets=auto default-pm=auto fabric=ibv generator=ninja arch=linux-ubuntu22.04-neoverse_v2
[+] ^ninja@1.11.1%gcc@12.3.0+re2c build_system=generic arch=linux-ubuntu22.04-neoverse_v2
[+] ^re2c@2.2%gcc@12.3.0 build_system=generic arch=linux-ubuntu22.04-neoverse_v2
[+] ^rdma-core@49.0%gcc@12.3.0~ipo~man_pages~pyverbs+static build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-ubuntu22.04-neoverse_v2
[+] ^libnl@3.3.0%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[e] ^bison@3.8.2%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[e] ^flex@2.6.4%gcc@12.3.0+lex~nls build_system=autotools patches=f8b85a0 arch=linux-ubuntu22.04-neoverse_v2
[+] ^mimalloc@2.1.2%gcc@12.3.0~build_tests~debug_full~ipo~local_dynamic_tls+override+padding~secure~see_asm~show_errors~skip_collect_on_exit~use_cxx~xmalloc build_system=cmake build_type=RelWithDebInfo generator=make libs=object,shared,static arch=linux-ubuntu22.04-neoverse_v2
[e] ^python@3.10.12%gcc@12.3.0+bz2+crypt+ctypes+dbm~debug+libxml2+lzma+nis~optimizations+pic+pyexpat~pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=0d98e93,7d40923,ebdca64,f2fd060 arch=linux-ubuntu22.04-neoverse_v2
[+] ^hpx-kokkos@0.4.0%gcc@12.3.0~cuda~ipo~rocm~sycl build_system=cmake build_type=RelWithDebInfo cxxstd=17 future_type=polling generator=make arch=linux-ubuntu22.04-neoverse_v2
[+] ^kokkos@4.2.00%gcc@12.3.0~aggressive_vectorization~compiler_warnings~cuda~debug~debug_bounds_check~debug_dualview_modify_check~deprecated_code~examples+hpx+hpx_async_dispatch~hwloc~ipo~memkind~numactl~openmp~openmptarget~pic~rocm+serial+shared~sycl~tests~threads~tuning~wrapper
build_system=cmake build_type=RelWithDebInfo cxxstd=17 generator=make intel_gpu_arch=none patches=145619e arch=linux-ubuntu22.04-neoverse_v2
[+] ^silo@4.11%gcc@12.3.0+fortran+fpzip+hdf5+hzip~mpi+pic+shared~silex build_system=autotools patches=251244d,451c4c5,a081263,eb2a3a0,fa050e0 arch=linux-ubuntu22.04-neoverse_v2
[e] ^autoconf@2.71%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[+] ^autoconf-archive@2023.02.20%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[e] ^automake@1.16.5%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[+] ^gnuconfig@2022-09-17%gcc@12.3.0 build_system=generic arch=linux-ubuntu22.04-neoverse_v2
[e] ^libtool@2.4.6%gcc@12.3.0 build_system=autotools arch=linux-ubuntu22.04-neoverse_v2
[e] ^m4@1.4.18%gcc@12.3.0+sigsegv build_system=autotools patches=3877ab5,fc9b616 arch=linux-ubuntu22.04-neoverse_v2
[e] ^perl@5.34.0%gcc@12.3.0~cpanm+opcode+open+shared+threads build_system=generic arch=linux-ubuntu22.04-neoverse_v2
[+] ^readline@8.2%gcc@12.3.0 build_system=autotools patches=bbf97f1 arch=linux-ubuntu22.04-neoverse_v2
[+] ^vc@1.4.1%gcc@12.3.0~ipo build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-ubuntu22.04-neoverse_v2
(So, looking at the arch flag in your Spec, one more difference would be the OS used)
Except for https://github.com/G-071/octotiger-spack/pull/5#discussion_r1498134026, I got that to work.
Hmm, cmake
rans fine, but at build time I will get:
[ 3%] Building CXX object CMakeFiles/hydrolib.dir/src/unitiger/hydro_impl/hydro_kernel_interface.cpp.o
/vast/home/junghans/octotiger-spack/spack/lib/spack/env/gcc/g++ -DCPPUDDLE_HAVE_HPX -DCPPUDDLE_HAVE_HPX_AWARE_ALLOCATORS -DCPPUDDLE_HAVE_HPX_MUTEX -DCPPUDDLE_HAVE_MAX_NUMBER_GPUS=1 -DCPPUDDLE_HAVE_NUMBER_
BUCKETS=128 -DH5_BUILT_AS_DYNAMIC_LIB -DHPX_KOKKOS_CUDA_FUTURE_TYPE=0 -DHPX_KOKKOS_SYCL_FUTURE_TYPE=0 -DHPX_LIBRARY_EXPORTS -DHPX_WITH_CUDA -DKOKKOS_DEPENDENCE -DOCTOTIGER_GRIDDIM=8 -DOCTOTIGER_HAVE_KOKKO
S -DOCTOTIGER_HAVE_STD_EXPERIMENTAL_SIMD -DOCTOTIGER_HAVE_VC -DOCTOTIGER_HYDRO_HOST_HPX_EXECUTOR -DOCTOTIGER_KOKKOS_HYDRO_TASKS=1 -DOCTOTIGER_KOKKOS_MONOPOLE_TASKS=1 -DOCTOTIGER_KOKKOS_MULTIPOLE_TASKS=1 -
DOCTOTIGER_KOKKOS_SIMD_SVE -DOCTOTIGER_MAX_NUMBER_FIELDS=15 -DOCTOTIGER_THETA_MINIMUM=0.34 -DSVE_LEN=128 -D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -D_LARGEFILE_SOURCE -D_POSIX_C_SOURCE=20
0809L -Dhydrolib_EXPORTS -I/vast/home/junghans/octotiger-spack/octotiger -I/vast/home/junghans/octotiger-spack/octotiger/spack-build/_deps/svesimd-src/include -isystem /vast/home/junghans/octotiger-spack/
spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/hpx-1.9.1-6ccd5erkj3dgdcbj34oe6y3g7tvpmag4/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/boost-
1.84.0-gtc6kjmh2hw3et27lfofdepq5y4ty2ud/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/hwloc-2.9.1-nsillyjc2h5xhv3k6hs7jvvs6tnivd5d/include -isyste
m /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/asio-1.22.2-r4jbde74qjc3c3dq26i7xulb7nkd6dtm/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/l
inux-rhel9-neoverse_n1/gcc-12.3.0/silo-4.11-f6niwvryvee6jtkldolycxme56ymqgji/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/hdf5-1.14.1-2-i2rtxbxar
f54wbpd5xsurtkd7l3mvuvh/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/zlib-1.2.13-xyz7izlqcwg44f4nkz55apfqzgrlsmo5/include -isystem /vast/home/jun
ghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/cppuddle-0.3.1-m3c5cwxelrbwfpwnnpzxbc35geol4snk/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-ne
overse_n1/gcc-12.3.0/vc-1.4.1-3sxlechnpz3743ldgoqbektzx4zu7nwf/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/kokkos-4.1.00-qdgwacoe5gbkjvcwmlmv2l7
26gavhvdm/include -isystem /vast/home/junghans/octotiger-spack/spack/opt/spack/linux-rhel9-neoverse_n1/gcc-12.3.0/hpx-kokkos-0.4.0-537vo6kmlgve6d6ltr2xhuljsatprs5p/include -Wno-cpp -fPIC -O2 -g -DNDEBUG -
std=gnu++20 -fPIC -msve-vector-bits=128 -pthread -ffp-contract=off -MD -MT CMakeFiles/hydrolib.dir/src/unitiger/hydro_impl/hydro_kernel_interface.cpp.o -MF CMakeFiles/hydrolib.dir/src/unitiger/hydro_impl/
hydro_kernel_interface.cpp.o.d -o CMakeFiles/hydrolib.dir/src/unitiger/hydro_impl/hydro_kernel_interface.cpp.o -c /vast/home/junghans/octotiger-spack/octotiger/src/unitiger/hydro_impl/hydro_kernel_interfa
ce.cpp
In file included from /vast/home/junghans/octotiger-spack/octotiger/octotiger/common_kernel/kokkos_simd.hpp:32,
from /vast/home/junghans/octotiger-spack/octotiger/octotiger/unitiger/hydro_impl/flux_kernel_templates.hpp:11,
from /vast/home/junghans/octotiger-spack/octotiger/octotiger/unitiger/hydro_impl/hydro_kokkos_kernel.hpp:22,
from /vast/home/junghans/octotiger-spack/octotiger/src/unitiger/hydro_impl/hydro_kernel_interface.cpp:16:
/vast/home/junghans/octotiger-spack/octotiger/octotiger/common_kernel/std_simd.hpp:51:28: error: 'sve' has not been declared
51 | namespace SIMD_NAMESPACE = sve::experimental::parallelism_v2;
| ^~~
this is with STEllAR-GROUP/octotiger@f42d82d3b0564e722be665c0169ddb04b9d74049 (which has allow_different_SVE_lengths
) and eaef1fb8df5296d10816152319b245f0250b0760 and spack/spack@0960f691a1ad80beeb32a95ea77edcf2f3fc79cc.
I suspect that for some reason __ARM_FEATURE_SVE is not set (since the include of the sve header work but the namespace therein is not defined afterwards).
Could you try the Octo-Tiger branch sve_build_improvements? It won't fix anything, but it'll give a more verbose error message if __ARM_FEATURE_SVE is actually undefined.
I think I ran into your new trap:
In file included from /vast/home/junghans/octotiger-spack/octotiger/octotiger/common_kernel/kokkos_simd.hpp:32,
from /vast/home/junghans/octotiger-spack/octotiger/octotiger/unitiger/hydro_impl/flux_kernel_templates.hpp:11,
from /vast/home/junghans/octotiger-spack/octotiger/octotiger/unitiger/hydro_impl/hydro_kokkos_kernel.hpp:22,
from /vast/home/junghans/octotiger-spack/octotiger/src/unitiger/hydro_impl/hydro_kernel_interface.cpp:16:
/vast/home/junghans/octotiger-spack/octotiger/octotiger/common_kernel/std_simd.hpp:49:2: error: #error "SVE Kokkos kernels are specified explicitly but build is without SVE support (__ARM_FEATURE_SVE not defined). Check architecture flags!"
49 | #error "SVE Kokkos kernels are specified explicitly but build is without SVE support (__ARM_FEATURE_SVE not defined). Check architecture flags!"
| ^~~~~
Closed as these changes will be merged as part a of #5
This PR adds the
sve_length
variant needed to make use of Octo-Tiger PR STEllAR-GROUP/octotiger#484. While I was at it, I also changed the cxx20 variant into a cxxstd variant as mentioned in #3 .@junghans Could you try this PR with
spack dev-build --until build --fresh octotiger@develop +kokkos simd_library=STD simd_extension=SVE sve_length=256 cxxstd=20 (... plus whatever else you needed in the spec)
and with STEllAR-GROUP/octotiger#484 checkout out in the octotiger source directory? Not quite sure if it works as intended on the target machine yet, but at least it seems to worksve_length=512
on the A64Fx machine I tested it on.