Failed to compile for GPU

[W-Wuxian]

Description

I got unexpected compiler errors while building Smilei 5.1 for omptask + GPU support. Where is my NVIDIA-SMI version report: 555.42.06 Driver Version: 555.42.06 CUDA Version: 12.5 I attached the makefile error log (makelog.log) make -j 12 machine="MyMachineFile" config="verbose omptasks gpu_nvidia" 2>&1 | tee makelog.log but most errors or warnings are like the following:

nvc++-Warning-No files to process
nvc++-Warning-Unknown switch: -Kfast
error: statement may not appear in a constexpr function
error: a constexpr function must contain exactly one return statement
error: identifier "__syncthreads" is undefined
error: identifier "__syncwarp" is undefined
ect.
error: identifier "threadIdx" is undefined
error: identifier "__cvta_generic_to_shared" is undefined
ect.

With config="verbose omptasks" make step ends without failure.

Steps to reproduce the problem

Loading modules

module purge
module load phdf5/nvhpc/24.7/1.14.3 nvhpc-hpcx-cuda12/nvhpc-hpcx-cuda12

Loaded modules: 1) phdf5/nvhpc/24.7/1.14.3 2) hpcx 3) nvhpc-hpcx-cuda12/nvhpc-hpcx-cuda12

Setting variables:

export SMILEICXX=nvc++
export GPU_COMPILER=nvcc
export BUILD_DIR=/opt/apps/libsoft/smilei/nvhpc/5.1
export PYTHONEXE=/opt/apps/miniconda3/env/root/envs/smilei_gpu_env/bin/python
export HDF5_ROOT_DIR=/opt/apps/fileformat/phdf5/nvhpc/1.14.3
export CXXFLAGS="-O3 -march=cascadelake -mtune=cascadelake -noswitcherror"
export OPENMP_FLAG="-fopenmp -D_OMP"

Compilation configuration

make -j 12 machine="MyMachineFile" config="verbose omptasks gpu_nvidia" 2>&1 | tee makelog.log

Parameters

• Smilei version: 5.1
• scripts/compile_tools/machine/MyMachineFile

  
  # Machine file
  # __________________________________________________________
  SMILEICXX_DEPS = g++
  # Options pour nvcxx (Compilateur C++ de NVHP)
  CXXFLAGS += -w -v
  CXXFLAGS += -tp=cascadelake -gpu=cc50 -acc=gpu -std=c++14  -lcurand -cudalib=curand
  CXXFLAGS += -Minfo=accel
  CXXFLAGS += -D__GCC_ATOMIC_TEST_AND_SET_TRUEVAL=1

Options pour NVCC (Compilateur CUDA)

GPU_COMPILER_FLAGS += -O3 --std c++14 -arch=sm_50 -v GPU_COMPILER_FLAGS += --expt-relaxed-constexpr

LDFLAGS += -acc=gpu -gpu=cc50 -v -std=c++14 -cudalib=curand -lcudart -lcurand -lacccuda

- `make -j 12 make -j 12 machine="MyMachineFile" config="verbose omptasks gpu_nvidia"`
- Computing resources
  - type of computer : Cluster NUMA node + 1 GPU Quadro M5000
  - OS : Linux Ubuntu 22.04.4 LTS
  - NVIDIA-SMI 555.42.06              Driver Version: 555.42.06      CUDA Version: 12.5  
  - C++ compiler g++ (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
  - MPI version:
    +  `mpic++ --version`:
```bash
nvc++ 24.7-0 64-bit target on x86-64 Linux -tp cascadelake 
NVIDIA Compilers and Tools
Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

• mpic++ -show:

  nvc++ -I/opt/apps/nvidia-hpc-sdk/24.7/Linux_x86_64/24.7/comm_libs/12.5/hpcx/hpcx-2.19/ompi/include -I/opt/apps/nvidia-hpc-sdk/24.7/Linux_x86_64/24.7/comm_libs/12.5/hpcx/hpcx-2.19/ompi/include/openmpi -I/opt/apps/nvidia-hpc-sdk/24.7/Linux_x86_64/24.7/comm_libs/12.5/hpcx/hpcx-2.19/ompi/include/openmpi/opal/mca/hwloc/hwloc201/hwloc/include -I/opt/apps/nvidia-hpc-sdk/24.7/Linux_x86_64/24.7/comm_libs/12.5/hpcx/hpcx-2.19/ompi/include/openmpi/opal/mca/event/libevent2022/libevent -I/opt/apps/nvidia-hpc-sdk/24.7/Linux_x86_64/24.7/comm_libs/12.5/hpcx/hpcx-2.19/ompi/include/openmpi/opal/mca/event/libevent2022/libevent/include -pthread -L/opt/apps/nvidia-hpc-sdk/24.7/Linux_x86_64/24.7/comm_libs/12.5/hpcx/hpcx-2.19/ompi/lib -Wl,-rpath -Wl,/opt/apps/nvidia-hpc-sdk/24.7/Linux_x86_64/24.7/comm_libs/12.5/hpcx/hpcx-2.19/ompi/lib -Wl,--enable-new-dtags -lmpi

• HDF5 version (h5cc --version):

  gcc (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
  Copyright (C) 2022 Free Software Foundation, Inc.
  This is free software; see the source for copying conditions.  There is NO
  warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

• Python version (python --version) : Python 3.12.4
• CXXFLAGS= -noswitcherror

[mccoys]

Please do not use omptasks. These are incompatible with gpu.

[charlesprouveur]

This is not a bug. As said above omptasks has nothing to do with the GPU implementation, please read the documentation on GPU compilation.

[W-Wuxian]

Hi, please review the attached makelog2.log as a result of the following configuration:

conda activate smilei_env
module load nvhpc-hpcx-cuda12/ phdf5/nvhpc/
export GPU_COMPILER=nvcc
export SMILEICXX=nvc++
export BUILD_DIR=/opt/apps/libsoft/smilei/nvhpc/5.1
export PYTHONEXE=/opt/apps/miniconda3/env/root/envs/smilei_gpu_env/bin/python
export HDF5_ROOT_DIR=/opt/apps/fileformat/phdf5/nvhpc/1.14.3

export CXXFLAGS="-O3 -march=cascadelake -mtune=cascadelake -noswitcherror -w -v -tp=cascadelake -gpu=cc50 -acc=gpu -std=c++14  -lcurand -cudalib=curand -Minfo=accel -D__GCC_ATOMIC_TEST_AND_SET_TRUEVAL=1"
export GPU_COMPILER_FLAGS="-O3 --std c++14 -arch=sm_50 -v --expt-relaxed-constexpr"
export LDFLAGS="-acc=gpu -gpu=cc50 -v -std=c++14 -cudalib=curand -lcudart -lcurand -lacccuda"
export SMILEICXX_DEPS=g++

make -j 12 config="verbose noopenmp gpu_nvidia" 2>&1 | tee makelog2.log

I still get the same errors as previously.