Closed pvelesko closed 3 years ago
Did you implement your own AoSoA structure in this case?
No, I just git pull and compile
On Thu, Jun 11, 2020, 12:16 AM Stuart Slattery notifications@github.com wrote:
Did you implement your own AoSoA structure in this case?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ECP-copa/Cabana/issues/256#issuecomment-642272074, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCJBQOLNTBYEMUNIROBB7TRV7Z3BANCNFSM4N2UXAKQ .
Weird - can I see how you built Cabana? Also did all of the Cabana unit tests build/pass?
#!/bin/bash
module load spack
module load cmake/3.17.0-gcc-9.3.0-5dgh2gv
module load kokkos/icpx
#source /soft/compilers/intel-2019/parallel_studio_xe_2019/psxevars.sh intel64
module load omp
echo "Using Kokkos: ${KOKKOS_HOME}"
export BUILD=build_kokkos_icpx
export CABANA_HOME=`pwd`
export CABANA_BUILD=`pwd`/${BUILD}
rm -f CMakeCache.txt
rm -rf ${CABANA_BUILD}
mkdir -p ${CABANA_BUILD}/install
cd ${CABANA_BUILD}
cmake ${CABANA_HOME} \
-DCMAKE_C_COMPILER=icpx \
-DCMAKE_CXX_COMPILER=icx \
-DCMAKE_CXX_FLAGS="-D_GLIBCXX_USE_CXX11_ABI=0 -lstdc++" \
-DCMAKE_BUILD_TYPE="Release" \
-DCMAKE_PREFIX_PATH=$KOKKOS_HOME/lib64/cmake/Kokkos \
-DCMAKE_INSTALL_PREFIX=${CABANA_BUILD}/install \
-DCabana_ENABLE_CAJITA=OFF \
-DCabana_ENABLE_TESTING=ON
#-DCabana_ENABLE_EXAMPLES=ON
make -j 20
make install
Unit tests:
[pvelesko@skylake06 build_kokkos_icpx]$ ctest
Test project /home/pvelesko/local/cabana/build_kokkos_icpx
Start 1: Cabana_Version_test
1/28 Test #1: Cabana_Version_test ................... Passed 0.04 sec
Start 2: Cabana_Index_test
2/28 Test #2: Cabana_Index_test ..................... Passed 0.08 sec
Start 3: Cabana_CartesianGrid_test
3/28 Test #3: Cabana_CartesianGrid_test ............. Passed 0.06 sec
Start 4: Cabana_SoA_test
4/28 Test #4: Cabana_SoA_test ....................... Passed 0.06 sec
Start 5: Cabana_AoSoA_test_SERIAL
5/28 Test #5: Cabana_AoSoA_test_SERIAL .............. Passed 0.10 sec
Start 6: Cabana_Slice_test_SERIAL
6/28 Test #6: Cabana_Slice_test_SERIAL .............. Passed 0.07 sec
Start 7: Cabana_DeepCopy_test_SERIAL
7/28 Test #7: Cabana_DeepCopy_test_SERIAL ........... Passed 0.08 sec
Start 8: Cabana_Tuple_test_SERIAL
8/28 Test #8: Cabana_Tuple_test_SERIAL .............. Passed 0.04 sec
Start 9: Cabana_Sort_test_SERIAL
9/28 Test #9: Cabana_Sort_test_SERIAL ............... Passed 0.09 sec
Start 10: Cabana_LinkedCellList_test_SERIAL
10/28 Test #10: Cabana_LinkedCellList_test_SERIAL ..... Passed 0.06 sec
Start 11: Cabana_NeighborList_test_SERIAL
11/28 Test #11: Cabana_NeighborList_test_SERIAL ....... Passed 0.56 sec
Start 12: Cabana_Parallel_test_SERIAL
12/28 Test #12: Cabana_Parallel_test_SERIAL ........... Passed 0.07 sec
Start 13: Cabana_AoSoA_test_OPENMP_1
13/28 Test #13: Cabana_AoSoA_test_OPENMP_1 ............ Passed 0.09 sec
Start 14: Cabana_AoSoA_test_OPENMP_2
14/28 Test #14: Cabana_AoSoA_test_OPENMP_2 ............ Passed 0.03 sec
Start 15: Cabana_Slice_test_OPENMP_1
15/28 Test #15: Cabana_Slice_test_OPENMP_1 ............ Passed 0.09 sec
Start 16: Cabana_Slice_test_OPENMP_2
16/28 Test #16: Cabana_Slice_test_OPENMP_2 ............ Passed 0.03 sec
Start 17: Cabana_DeepCopy_test_OPENMP_1
17/28 Test #17: Cabana_DeepCopy_test_OPENMP_1 ......... Passed 0.08 sec
Start 18: Cabana_DeepCopy_test_OPENMP_2
18/28 Test #18: Cabana_DeepCopy_test_OPENMP_2 ......... Passed 0.03 sec
Start 19: Cabana_Tuple_test_OPENMP_1
19/28 Test #19: Cabana_Tuple_test_OPENMP_1 ............ Passed 0.03 sec
Start 20: Cabana_Tuple_test_OPENMP_2
20/28 Test #20: Cabana_Tuple_test_OPENMP_2 ............ Passed 0.05 sec
Start 21: Cabana_Sort_test_OPENMP_1
21/28 Test #21: Cabana_Sort_test_OPENMP_1 ............. Passed 0.11 sec
Start 22: Cabana_Sort_test_OPENMP_2
22/28 Test #22: Cabana_Sort_test_OPENMP_2 ............. Passed 0.10 sec
Start 23: Cabana_LinkedCellList_test_OPENMP_1
23/28 Test #23: Cabana_LinkedCellList_test_OPENMP_1 ... Passed 0.10 sec
Start 24: Cabana_LinkedCellList_test_OPENMP_2
24/28 Test #24: Cabana_LinkedCellList_test_OPENMP_2 ... Passed 0.02 sec
Start 25: Cabana_NeighborList_test_OPENMP_1
25/28 Test #25: Cabana_NeighborList_test_OPENMP_1 ..... Passed 0.45 sec
Start 26: Cabana_NeighborList_test_OPENMP_2
26/28 Test #26: Cabana_NeighborList_test_OPENMP_2 ..... Passed 0.26 sec
Start 27: Cabana_Parallel_test_OPENMP_1
27/28 Test #27: Cabana_Parallel_test_OPENMP_1 ......... Passed 0.65 sec
Start 28: Cabana_Parallel_test_OPENMP_2
28/28 Test #28: Cabana_Parallel_test_OPENMP_2 ......... Passed 0.07 sec
100% tests passed, 0 tests failed out of 28
Total Test time (real) = 3.56 sec
@pvelesko couldn't you build Cabana with Spack as well?
@junghans I have never used spack, I just found some modules named spack which gave me access to cmake. Either way, that is not a long-term solution. I'd like to figure out what is going on here.
@pvelesko Do you include Cabana_Core.hpp
or just individual header files?
cpp/cpp_species.hpp:#include <Cabana_AoSoA.hpp>
cpp/cpp_species.hpp:#include <Cabana_DeepCopy.hpp>
Those includes should be fine. Is it possible that an AoSoA
isn't getting passed into the deep_copy
function you are calling? Does the compiler tell you what the type of DstAoSoA
is in the error output? Usually on a bad template deduction it will tell you what type you passed in.
Also @pvelesko can you share the code that you are building?
Everything is building fine with icpc
for me, I guess this is an icpx
specific issue? We'll probably need a reproducer..
I filed another ticket for icpc. It doesn't build at all with icpc. And I did enable c14 in kokkos.
On Thu, Jun 11, 2020, 11:04 PM Robert Bird notifications@github.com wrote:
Everything building fine with icpc, I guess this is an icpx specific issue? We'll probably need a reproducer..
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ECP-copa/Cabana/issues/256#issuecomment-642900349, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACCJBQOCNQFFEISIKVUYYW3RWE2FZANCNFSM4N2UXAKQ .
Turns out there was an older version of Cabana in a different place on my system that was getting picked up when compiling the application but not Cabana itself. Sorry!
No idea why this happens.. Looking at the file, I see the include but somehow it fails with the following error:
gmake[1]: Leaving directory
/home/pvelesko/XGC-Devel/XGC_core' /usr/bin/gmake -C ./XGC_core/cpp objects_cpp gmake[1]: Entering directory/home/pvelesko/XGC-Devel/XGC_core/cpp' mpiicpc -Wall -qopenmp -std=c++14 -lstdc++ -DPSPLINE -DUSE_BICUB_MOD -DUSE_ONE_D_I_CUB_MOD -DV_PERP -DSOLVERLU -DUSE_INQUIRE_DIRECTORY -DITER_GRID -DCAM_TIMERS -DXGC1 -DUSE_CAB -DUSE_CAB_CPP -DUSE_CAB_OMP=1 -DUSE_CAB_GPU=0 -DUSE_ARRAY_REPLICATION -DSIMD_SIZE=32 -DVEC_LEN=32 -I/home/pvelesko/local/kokkos_mainline/build_icpx/install/include -I/home/pvelesko/local/cabana/build_kokkos_icpx/install/include -I/home/pvelesko -I/home/pvelesko/local/intel/x86/include -I/home/pvelesko/local/kokkos_mainline/build_icpx/install/include -I/home/pvelesko/local/cabana/build_kokkos_icpx/install/include -I/home/pvelesko /home/pvelesko/XGC-Devel/xgc_build -c -o /home/pvelesko/XGC-Devel/xgc_build/cpp_scatter.o cpp_scatter.cpp In file included from cpp_species.hpp(4), from cpp_transpose_send.hpp(4), from cpp_scatter.cpp(5): /home/pvelesko/local/cabana/build_kokkos_icpx/install/include/Cabana_DeepCopy.hpp(156): error: identifier "is_aosoa" is undefined typename std::enable_if<( is_aosoa<DstAoSoA>::value &&