pmarguinaud
commented
3 months ago Maybe we could just remove these lines in field_api cmake configuration :
34 if( fckit_FOUND AND fckit_HAVE_FCKIT_VENV )
35 set( FYPP ${FCKIT_VENV_EXE} -m fypp )Closed pmarguinaud closed 2 months ago
pmarguinaud
commented
3 months ago Maybe we could just remove these lines in field_api cmake configuration :
34 if( fckit_FOUND AND fckit_HAVE_FCKIT_VENV )
35 set( FYPP ${FCKIT_VENV_EXE} -m fypp )
awnawab
commented
3 months ago Hi @pmarguinaud. The easiest way to resolve this would be to upgrade to fckit 0.13.0 (currently you are using 0.9.5) and build fckit with the argument -DENABLE_FCKIT_VENV=ON. Please let me know if you are able to use a newer version of fckit and if this works, otherwise we will need a workaround.
The problem actually doesn't lie in the fact that field_api picks up fckit, but rather that fckit exports the variable FYPP = fckit-eval.sh. fckit-eval.sh does some string sanitisation that strips the suffix definitions. This is worked around with fckit 0.13.0 and lines 34-35 in the top-level field_api cmakelists.txt.
For context: fckit 0.13.0 ships with a minimal yaml parser + fypp, so that we can use these on bare bones systems that don't have pyyaml (and we are unable to run pip3 install)
pmarguinaud
commented
3 months ago Hello Ahmad,
An thank you for your answer and explanations.
Unfortunately, I have the same problem with version 0.130.0 of fckit ; please note that I cannot enable FCKIT_VENV, as we only have python 3.6 on our system :
+ set -e
+ cmake -Wno-deprecated -Wno-dev -DCMAKE_C_COMPILER=/home/gmap/mrpm/marguina/gpupack/support/wrap/NVHPC2405/mpicc '-DCMAKE_C_FLAGS= -mp -fPIC -Minfo -gopt -Mlarge_arrays -Mlist -traceback -Mnofma -g -O0 -cuda -acc=gpu -O0 -gopt -gpu=cc70,cc80' -DCMAKE_Fortran_COMPILER=/home/gmap/mrpm/marguina/gpupack/support/wrap/NVHPC2405/mpif90 '-DCMAKE_Fortran_FLAGS= -mp -fPIC -Minfo -gopt -Mlarge_arrays -Mlist -traceback -Mnofma -Mbyteswapio -Mbackslash -Mstack_arrays -g -O0 -cuda -acc=gpu -O0 -gopt -gpu=cc70,cc80 -lstdc++' -DCMAKE_BUILD_TYPE=NONE -DFYPP_PATH=/home/gmap/mrpm/marguina/gpupack/install/common/fypp/bin/fypp -DENABLE_SINGLE_PRECISION=OFF -DENABLE_DOUBLE_PRECISION=ON -DENABLE_ACC=ON -DCMAKE_CUDA_COMPILER=/home/gmap/mrpm/marguina/gpupack/support/wrap/NVHPC2405/ccu -DCMAKE_INSTALL_PREFIX=/home/gmap/mrpm/marguina/jetpack/install/FieldAPI -Decbuild_ROOT=/home/gmap/mrpm/marguina/jetpack/install/ecSDK -Deckit_ROOT=/home/gmap/mrpm/marguina/jetpack/install/ecSDK -Dfckit_ROOT=/home/gmap/mrpm/marguina/jetpack/install/ecSDK -Doops_ROOT=/home/gmap/mrpm/marguina/jetpack/install/OOPS -Dfiat_ROOT=/home/gmap/mrpm/marguina/jetpack/install/Fiat -Dectrans_ROOT=/home/gmap/mrpm/marguina/jetpack/install/Ectrans /home/gmap/mrpm/marguina/jetpack/IAL-external/FieldAPI/field_api
-- The Fortran compiler identification is NVHPC 24.5.0
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /home/gmap/mrpm/marguina/gpupack/support/wrap/NVHPC2405/mpif90 - skipped
-- ecbuild 3.7.0 /home/gmap/mrpm/marguina/jetpack/install/ecSDK/share/ecbuild/cmake
-- cmake 3.28.2 /home/gmap/mrpm/marguina/gpupack/install/common/cmake/bin/cmake
-- ---------------------------------------------------------
-- Found Git: /home/gmap/mrpm/marguina/bin/git (found version "2.27.0")
-- The C compiler identification is NVHPC 24.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/gmap/mrpm/marguina/gpupack/support/wrap/NVHPC2405/mpicc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Check size of off_t
-- Check size of off_t - done
-- ---------------------------------------------------------
-- ---------------------------------------------------------
-- [32m[field_api] (0.3.0) [80f0c45][m
-- Feature TESTS enabled
-- Found OpenMP_Fortran: -mp (found version "5.1")
-- Found OpenMP: TRUE (found version "5.1") found components: Fortran
-- field_api FOUND OpenMP
-- field_api FOUND fckit: /home/gmap/mrpm/marguina/jetpack/install/ecSDK/lib64/cmake/fckit (found version "0.13.0")
-- FCKIT_LIBRARIES : [fckit]
-- Found OpenACC_C: -acc
-- Found OpenACC_Fortran: -acc
-- Feature ACC enabled
-- Performing Test field_api_Fortran_FLAG_TEST_1
-- Performing Test field_api_Fortran_FLAG_TEST_1 - Success
-- Added Fortran flag [-Mlarge_arrays]
-- Performing Test field_api_Fortran_FLAG_TEST_2
-- Performing Test field_api_Fortran_FLAG_TEST_2 - Success
-- Added Fortran flag [-gopt]
-- Performing Test field_api_Fortran_FLAG_TEST_3
-- Performing Test field_api_Fortran_FLAG_TEST_3 - Success
-- Added Fortran flag [-Minfo=accel,all,ccff] to build type DEBUG
-- Feature DOUBLE_PRECISION enabled
-- field_api FOUND fiat: /home/gmap/mrpm/marguina/jetpack/install/Fiat/lib64/cmake/fiat (found version "1.2.0")
-- Feature CUDA enabled
-- Feature BUDDY_MALLOC enabled
-- Looking for backtrace
-- Looking for backtrace - found
CMake Warning at /home/gmap/mrpm/marguina/jetpack/install/ecSDK/share/ecbuild/cmake/ecbuild_log.cmake:162 (message):
[33mWARN - OpenACC builds force static linking.[m
Call Stack (most recent call first):
CMakeLists.txt:155 (ecbuild_warn)
-- Feature TESTS enabled
-- Configuring done (18.5s)
-- Generating done (2.4s)
CMake Warning:
Manually-specified variables were not used by the project:
FYPP_PATH
ectrans_ROOT
oops_ROOT
-- Build files have been written to: /home/gmap/mrpm/marguina/jetpack/build/FieldAPI/field_api
+ make -j16 install
[ 0%] Generating host_alloc_module.F90
[ 1%] Generating dev_alloc_module.F90
[ 1%] Generating field_1im_access_module.F90
[ 2%] Generating field_1im_array_module.F90
[ 2%] Generating field_1im_data_module.F90
[ 2%] Generating field_1im_array_util_module.F90
[ 2%] Generating field_1im_factory_module.F90
[ 2%] Generating field_1im_gang_module.F90
[ 2%] Generating field_1im_gather_module.F90
[ 2%] Generating field_1im_module.F90
[ 2%] Building Fortran object tests/CMakeFiles/check_gpu_num.x.dir/check_gpu_num.F90.o
[ 2%] Generating field_1im_shuffle_module.F90
[ 2%] Generating field_1im_util_module.F90
[ 2%] Generating field_1lm_access_module.F90
[ 2%] Generating field_1lm_array_module.F90
[ 3%] Generating field_1lm_array_util_module.F90
error: exception at evaluating 'IM' in definition for 'SUFF' [FyppFatalError]
error: name 'IM' is not defined [NameError]
error: exception at evaluating 'IM' in definition for 'SUFF' [FyppFatalError]
error: name 'IM' is not defined [NameError]
error: exception at evaluating 'IM' in definition for 'SUFF' [FyppFatalError]
error: name 'IM' is not defined [NameError]
error: exception at evaluating 'IM' in definition for 'SUFF' [FyppFatalError]
error: name 'IM' is not defined [NameError]
error: exception at evaluating 'IM' in definition for 'SUFF' [FyppFatalError]
error: name 'IM' is not defined [NameError]
error: exception at evaluating 'IM' in definition for 'SUFF' [FyppFatalError]
error: name 'IM' is not defined [NameError]
error: exception at evaluating 'IM' in definition for 'SUFF' [FyppFatalError]
error: name 'IM' is not defined [NameError]
error: exception at evaluating 'IM' in definition for 'SUFF' [FyppFatalError]
error: name 'IM' is not defined [NameError]
make[2]: *** [field_1im_access_module.F90] Error 1
make[2]: *** Deleting file `field_1im_access_module.F90'
error: exception at evaluating 'IM' in definition for 'SUFF' [FyppFatalError]
error: name 'IM' is not defined [NameError]
error: exception at evaluating 'IM' in definition for 'SUFF' [FyppFatalError]
error: name 'IM' is not defined [NameError]
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [field_1im_array_module.F90] Error 1
make[2]: *** Deleting file `field_1im_array_module.F90'
error: exception at evaluating 'LM' in definition for 'SUFF' [FyppFatalError]
error: name 'LM' is not defined [NameError]
make[2]: *** [field_1im_data_module.F90] Error 1
make[2]: *** Deleting file `field_1im_data_module.F90'
make[2]: *** [field_1im_array_util_module.F90] Error 1
make[2]: *** Deleting file `field_1im_array_util_module.F90'
error: exception at evaluating 'LM' in definition for 'SUFF' [FyppFatalError]
error: name 'LM' is not defined [NameError]
make[2]: *** [field_1im_factory_module.F90] Error 1
make[2]: *** Deleting file `field_1im_factory_module.F90'
error: exception at evaluating 'LM' in definition for 'SUFF' [FyppFatalError]
error: name 'LM' is not defined [NameError]
make[2]: *** [field_1im_gang_module.F90] Error 1
make[2]: *** Deleting file `field_1im_gang_module.F90'
make[2]: *** [field_1im_gather_module.F90] Error 1
make[2]: *** Deleting file `field_1im_gather_module.F90'
make[2]: *** [field_1im_module.F90] Error 1
make[2]: *** Deleting file `field_1im_module.F90'
make[2]: *** [field_1im_shuffle_module.F90] Error 1
make[2]: *** Deleting file `field_1im_shuffle_module.F90'
make[2]: *** [field_1im_util_module.F90] Error 1
make[2]: *** Deleting file `field_1im_util_module.F90'
make[2]: *** [field_1lm_access_module.F90] Error 1
make[2]: *** Deleting file `field_1lm_access_module.F90'
make[2]: *** [field_1lm_array_util_module.F90] Error 1
make[2]: *** Deleting file `field_1lm_array_util_module.F90'
make[2]: *** [field_1lm_array_module.F90] Error 1
make[2]: *** Deleting file `field_1lm_array_module.F90'
make[1]: *** [CMakeFiles/field_api_dp.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 3%] Linking Fortran executable check_gpu_num.x
[ 3%] Built target check_gpu_num.x
make: *** [all] Error 2Maybe we could add a test on fckit version ?
awnawab
commented
3 months ago Right, yes it won't work without FCKIT_VENV. Our system's default python is also 3.6. Considering python 3.8 is almost 5 years old, this is quite unfortunate.
The cmake output suggests you are using an installed fckit build, in which case the only mechanism for fckit to export variables to field_api is via ecbuild_find_package. So commenting out ecbuild_find_package(fckit) should do the trick. Failing that, you can also try adding unset(FYPP) before find_program(FYPP fypp).
I hope these workarounds can unblock you for the moment. If there is no easy way to access a newer version of python during build time (maybe your admins can make python3.8 available as a module), then I can build in a cleaner workaround once I am back from paternity leave in a couple of weeks.
pmarguinaud
commented
3 months ago Thank you Ahmad, and congratulations for being a father. Much more difficult than fixing field_api.
I will wait for you to come back to work.
awnawab
commented
2 months ago Hi @pmarguinaud. I am back at work this week. Were you able to use a newer version of python and use FCKIT_VENV or should I implement a workaround in FIELD_API?
pmarguinaud
commented
2 months ago Hello Ahmad,
Unfortunately, it is not possible for us to use a newer version of python, so I would like to have an option in field_api so that we can avoid using FCKIT_ENV even when fckit is available.
Thank you for your help
Hello,
Our build system adds automatically the fckit project to all Fortran libraries; unfortunately, we have an issue with field_api :
Is this expected ? How do I solve this problem ?