search

entity-toolkit / entity

New generation astrophysical plasma simulation code with CPU/GPU portability
https://entity-toolkit.github.io/wiki/
Other
28 stars 2 forks source link

compile error #33

Closed StaticObserver closed 9 months ago

StaticObserver commented 9 months ago

Error occurs when building with GPU codes when only built with openmp, it works fine, also works for gpu on other computers, I don't get it

ibentity-framework.a(output_flds.cpp.o): In function ntt::OutputField::initialize(ntt::SimulationEngine const&)': tmpxft_001fda24_00000000-6_output_flds.cudafe1.cpp:(.text+0x74): multiple definition ofntt::OutputField::initialize(ntt::SimulationEngine const&)' libentity-framework.a(engines.cpp.o):tmpxft_001fd9e9_00000000-6_engines.cudafe1.cpp:(.text+0x92): first defined here libentity-framework.a(output_flds.cpp.o): In function ntt::OutputField::name[abi:cxx11](int const&) const': tmpxft_001fda24_00000000-6_output_flds.cudafe1.cpp:(.text+0x14c0): multiple definition ofntt::OutputField::name[abi:cxx11](int const&) const' libentity-framework.a(engines.cpp.o):tmpxft_001fd9e9_00000000-6_engines.cudafe1.cpp:(.text+0x14de): first defined here libentity-framework.a(fields.cpp.o): In function ntt::Fields<(ntt::Dimension)1, (ntt::SimulationEngine)2>::Fields(std::vector<unsigned int, std::allocator<unsigned int> >)': tmpxft_001fd9b6_00000000-6_fields.cudafe1.cpp:(.text+0x92): multiple definition ofntt::Fields<(ntt::Dimension)1, (ntt::SimulationEngine)2>::Fields(std::vector<unsigned int, std::allocator >)' libentity-framework.a(engines.cpp.o):tmpxft_001fd9e9_00000000-6_engines.cudafe1.cpp:(.text+0x16e4): first defined here libentity-framework.a(fields.cpp.o): In function ntt::Fields<(ntt::Dimension)1, (ntt::SimulationEngine)2>::Fields(std::vector<unsigned int, std::allocator<unsigned int> >)': tmpxft_001fd9b6_00000000-6_fields.cudafe1.cpp:(.text+0x92): multiple definition ofntt::Fields<(ntt::Dimension)1, (ntt::SimulationEngine)2>::Fields(std::vector<unsigned int, std::allocator >)' libentity-framework.a(engines.cpp.o):tmpxft_001fd9e9_00000000-6_engines.cudafe1.cpp:(.text+0x16e4): first defined here libentity-framework.a(fields.cpp.o): In function `ntt::Fields<(ntt::Dimension)2, (ntt::SimulationEngine)2>::Fields(std::vector<unsigned int, std::allocator >)':

haykh commented 9 months ago

Could you give a bit more details? Which hardware are you compiling on? Does it find the kokkos/adios libraries? Which branch are you using? How do you configure the code (the cmake command)?

StaticObserver commented 9 months ago

I'm compiling on a hpc with Intel Xeon ICX Platinum 8358 cpu. cmake command is the following: cmake ../ -B build -D pgen=weibel -D engine=pic -D output=ON -D precision=double -DKokkos_ENABLE_OPENMP=ON -DKokkos_ENABLE_CUDA=ON

the kokkos has already been compiled before with : cmake ../ -DCMAKE_CXX_COMPILER=/dssg/home/acct-tdlmzn/tdlmzn-yangyangcai/sources/kokkos/bin/nvcc_wrapper -DCMAKE_INSTALL_PREFIX=/dssg/home/acct-tdlmzn/tdlmzn-yangyangcai/Kokkos -DKokkos_ENABLE_OPENMP=ON -DKokkos_ENABLE_CUDA=ON -DKokkos_ENABLE_CUDA_CONSTEXPR=ON -DKokkos_ENABLE_CUDA_LAMBDA=ON -DKokkos_ENABLE_CUDA_LDG_INTRINSIC=ON -DKokkos_ENABLE_CUDA_RELOCATABLE_DEVICE_CODE=ON -DKokkos_ARCH_AMPERE80=ON -DKokkos_ARCH_NATIVE=ON

when configuring entity, the outputs are:

-- The CXX compiler identification is GNU 12.3.0 -- The C compiler identification is GNU 12.3.0 -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /dssg/opt/icelake/linux-centos8-icelake/gcc-11.2.0/gcc-12.3.0-4kizoqayd6dyg5awzjbl7nv24d6ixrji/bin/g++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /dssg/opt/icelake/linux-centos8-icelake/gcc-11.2.0/gcc-12.3.0-4kizoqayd6dyg5awzjbl7nv24d6ixrji/bin/gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Release mode connect: Network is unreachable -- No internet connection. Fetching disabled. -- kokkos_launch_compiler is enabled globally. C++ compiler commands with -DKOKKOS_DEPENDENCE will be redirected to the appropriate compiler for Kokkos -- Kokkos found. -- fmt not found. Using as submodule. -- Version: 10.2.0 -- Build type: -- plog not found. Using as submodule. -- toml11 not found. Using as submodule. -- adios2 found. -- CUDA compiler: /dssg/opt/icelake/linux-centos8-icelake/gcc-11.2.0/cuda-12.1.1-xkp7tw76f32u7vlt3o2sbotrsxwnvrkn/bin/nvcc

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . /\ _ /\ \ . . \ \ \/_\ \ \ . . / \ / \ \ \/\/\ \ \ \/ /\ \/\ \ . . /\ //\ \/\ \ \ _\ \ \ \ _\ \ _\ \ . . \ _\ _\ _\ _\ _\ _\ ____ \/\\ . . \/__/\//\//\// \/_/\/_/ \// \/_/ . . /_/ . . \// . . . . v0.9.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Main configurations [1]

Compile configurations

Dependencies

Notes

[1] Set with cmake ... -D <FLAG>=<VALUE>, the default value : will be used unless the variable is explicitly set.

-- Configuring done (6.1s) -- Generating done (0.1s) -- Build files have been written to: /dssg/home/acct-tdlmzn/tdlmzn-yangyangcai/entity/weibel-test-gpu/build

StaticObserver commented 9 months ago

gcc version is 12.3.0 cuda version is 12.1.0

StaticObserver commented 9 months ago

I figured it out! It is the option -DKokkos_ENABLE_CUDA_RELOCATABLE_DEVICE_CODE=ON I used on compiling Kokkos that's causing the problem. I don't know the reason behind, but when I remove the option, the code entity compiles well.

haykh commented 9 months ago

that is weird. what GPU are you compiling this for? you're using ampere80, so i assume it's an A100?

a few notes:

  1. why are you using ../ to configure?

    cmake ../ -B build -D pgen=weibel -D engine=pic -D output=ON -D precision=double -DKokkos_ENABLE_OPENMP=ON -DKokkos_ENABLE_CUDA=ON

i think it's not an issue, but it might mess up some of the paths. simply do (from the root) cmake -B build ..., and then cmake --build build -j.

  1. you can use this tool to properly build/install kokkos. at some point, i will also release a spack module.

  2. Kokkos_ENABLE_CUDA_LAMBDA and Kokkos_ENABLE_CUDA_CONSTEXPR are enabled by default in latest kokkos.

  3. RDC that you enabled in your kokkos compilation is a way of embedding the host code around the device code (i think it determines the order in which the kernels are linked). are you sure it finds the right kokkos? if it doesn't, it actually downloads and compiles kokkos automatically. in general, if you're using a preinstalled kokkos, i would specify the kokkos path during configuration using cmake -B build ... -D Kokkos_DIR=/path/to/kokkos.

StaticObserver commented 9 months ago

I'am sure they found the kokkos I built in the system.
It is only the RDC option annoys your code. It would think there are mutiple definations of one function On my own PC. l let cmake download the externs and compile all the dependencies for me, then everything works.

haykh commented 9 months ago

ok, sounds good. marking this as resolved. feel free to reopen if any follow-up is necessary.