Closed alapha23 closed 6 years ago
Hi,
thanks for reporting this! To be honest, I've never tried to build Open-SURFEX under Windows so there could be some bugs. If with Cygwin we have the same case-insensitive Windows-flavored filenames, then the problem is clear for me.
If you check the list_sources
function in the CMakeLists.txt:49-58
, you could see that it tries all possible extensions to build a list of Fortran sources. And there are the both lower-case and upper-case extensions: set(endings "*.F90" "*.f90" "*.F" "*.f")
. So, under Windows it could add the same files twice because of case-insensitive file system. And this duplication causes the errors.
As a quick solution you could try to just remove remove lower-case extensions from the endings-list.
Hi! Thank you for the answer!
Modifying the Fortran sources to set(ending "*.F90" )
does solve the issue!
However, error message below happens at cmake --build .
, it seems the flag -DDISABLE_OS_CHECK=ON
is required.
Again when I tried to build with command cmake -DDISABLE_OS_CHECK=ON --build .
, the warning Manually-specified variables were not used by the project:
pops up and build stops.
Could you possibly suggest how to use that flag in CMake? Thank you very much!
CMake Error at cmake/ecbuild_log.cmake:191 (message):
CRITICAL - ecBuild is untested for this operating system: [CYGWIN]
-- refusing to continue. Disable this check with -DDISABLE_OS_CHECK=ON
This error comes from the grib-api
library. To solve this one, you could try to add -DDISABLE_OS_CHECK=ON
to the list of grib-api
options in the cmake/Dependencies.cmake:106-113
. If it doesn't help then you need to install grib-api
manually (either from source or precompiled) and set up the Open-SURFEX build with:
cmake .. -DBUILD_GRIB_API=OFF -DGRIB_API_DIR=<path to GRIB API> -DCMAKE_INSTALL_PREFIX=../_install
Thank you for the answer!!
Adding -DDISABLE_OS_CHECK=ON
in cmake/Dependencies.cmake
did solve the problem!!
However, I am having a hard time since there is a new issue related to directory path. I have been tweaking for quite a while but I don't seem to identify the exact problem. Could you possibly give some suggestions?
target pattern contains no '%'
pops up now at cmake --build .
.
The error message is as follows
[ 98%] Linking Fortran static library libsurfex.a
[ 98%] Built target surfex-static
CMakeFiles/sxpost.dir/build.make:83: *** target pattern contains no '%'. Stop.
make[1]: *** [CMakeFiles/Makefile2:72: CMakeFiles/sxpost.dir/all] Error 2
make: *** [Makefile:130: all] Error 2
While in CMakeFiles/sxpost.dir/build.make
, line 83 is SXPOST.exe: NetCDF::NetCDF_Fortran-NOTFOUND
SXPOST.exe: CMakeFiles/sxpost.dir/src/OFFLIN/sxpost.F90.o
SXPOST.exe: CMakeFiles/sxpost.dir/build.make
SXPOST.exe: libsurfex.a
SXPOST.exe: NetCDF::NetCDF_Fortran-NOTFOUND
SXPOST.exe: grib_api::grib_api_Fortran-NOTFOUND
SXPOST.exe: CMakeFiles/sxpost.dir/link.txt
In [CMakeFiles/Makefile2
, below section is line 71~75
CMakeFiles/sxpost.dir/rule: cmake_check_build_system
$(CMAKE_COMMAND) -E cmake_progress_start /home/alaph/gao/open-SURFEX/build/CMakeFiles 98
$(MAKE) -f CMakeFiles/Makefile2 CMakeFiles/sxpost.dir/all
$(CMAKE_COMMAND) -E cmake_progress_start /home/alaph/gao/open-SURFEX/build/CMakeFiles 0
.PHONY : CMakeFiles/sxpost.dir/rule
In Makefile from line 127~132
# The main all target
all: cmake_check_build_system
$(CMAKE_COMMAND) -E cmake_progress_start /home/alaph/gao/open-SURFEX/build/CMakeFiles /home/alaph/gao/open-SURFEX/build/CMakeFiles/progress.marks
$(MAKE) -f CMakeFiles/Makefile2 all
$(CMAKE_COMMAND) -E cmake_progress_start /home/alaph/gao/open-SURFEX/build/CMakeFiles 0
.PHONY : all
When I remove sxpost
from the targets in CMakeLists.txt
, the sxpost
in above error message would be replaced by prep
.
I saw several similar issues by googling, which mostly relates to directory path parsing.
In our case, the pathname in Makefile2 and build.make seem to be a legal pathname, which leaves me no clue on further debuging.
What do you think? In addition, is it okay if I keep puting questions here? Or it's better if I raise a new issue?
Thank you very much!
Hm, it looks like there's something wrong with libraries. These two lines doesn't look healthy:
SXPOST.exe: NetCDF::NetCDF_Fortran-NOTFOUND
SXPOST.exe: grib_api::grib_api_Fortran-NOTFOUND
Could you provide output from the cmake
configuration step? Were there any messages about missing targets? It would be nice also to see the contents of the auxiliary/lib
inside the build directory.
As a quick fix, since you have built all required libraries, you could try to re-configure the build system to use pre-built grib-api
and NetCDF
libraries in the following way:
cmake .. -DBUILD_NETCDF=NO -DBUILD_GRIB_API=NO -DNETCDF_DIR=auxiliary/ -DGRIB_API_DIR=auxiliary/ -DCMAKE_INSTALL_PREFIX=../_install
though, you could get a bunch of new errors.
It's fine to keep putting questions in this issue, as long as they are Cygwin-specific.
Thank you for the reply!
cmake .. -DCMAKE_INSTALL_PREFIX=../_install
, below message shows upFrom what I am seeing, there isn't any missed targets.
$ cmake .. -DCMAKE_INSTALL_PREFIX=../_install
-- The C compiler identification is GNU 7.3.0
-- The CXX compiler identification is GNU 7.3.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++.exe
-- Check for working CXX compiler: /usr/bin/c++.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The Fortran compiler identification is GNU 7.3.0
-- Check for working Fortran compiler: /usr/bin/gfortran.exe
-- Check for working Fortran compiler: /usr/bin/gfortran.exe -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether /usr/bin/gfortran.exe supports Fortran 90
-- Checking whether /usr/bin/gfortran.exe supports Fortran 90 -- yes
-- CMAKE_BUILD_TYPE is not set, switching to RELEASE
-- Using /home/alaph/gao/open-SURFEX/conf/config.GNU.cmake
-- Configuring done
-- Generating done
-- Build files have been written to: /home/alaph/gao/open-SURFEX/build
cmake .. -DBUILD_NETCDF=NO -DBUILD_GRIB_API=NO -DNETCDF_DIR=auxiliary/ -DGRIB_API_DIR=auxiliary/ -DCMAKE_INSTALL_PREFIX=../_install
-- Using /home/alaph/gao/open-SURFEX/conf/config.GNU.cmake
-- Failed to find NetCDF interface for F90
CMake Error at /usr/share/cmake-3.12.0-rc3/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
Could NOT find NetCDF (missing: NETCDF_LIBRARIES NETCDF_INCLUDE_DIRS
NETCDF_HAS_INTERFACES)
Call Stack (most recent call first):
/usr/share/cmake-3.12.0-rc3/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
cmake/FindNetCDF.cmake:155 (find_package_handle_standard_args)
cmake/Dependencies.cmake:97 (find_package)
CMakeLists.txt:46 (include)
I also tried to move auxiliary
folder elsewhere and re-configure it. The message includes other check but it stops at the same error.
auxiliary
This is my auxiliary.zip
Briefly, inside auxiliary/lib
,
If I could make a guess, the way Cygwin deals with pathname might affect the parsing pathname of NetCDF and grib-api. Hence, NetCDF and grib-api cannot be found---even though they are built successfully.
What do you think?
Thank you very much!
It seems that under Cygwin the libraries were put into the auxiliary/bin
with some cygwin-specific prefixes:
bin/cyggrib_api.dll
bin/cyggrib_api_f77.dll
bin/cyggrib_api_f90.dll
bin/cyghdf5-10.3.0.dll
bin/cyghdf5_hl-10.3.0.dll
bin/cygnetcdf-11.dll
bin/cygnetcdff-6.dll
You could try to make symlinks to point these dlls from bin
to lib
and rename them to have a common prefix:
bin/cyggrib_api.dll -> lib/libgrib_api.dll
bin/cyggrib_api_f77.dll -> lib/libgrib_api_f77.dll
bin/cyggrib_api_f90.dll -> lib/libgrib_api_f90.dll
bin/cyghdf5-10.3.0.dll -> lib/libhdf5-10.3.0.dll
bin/cyghdf5_hl-10.3.0.dll -> lib/libhdf5_hl-10.3.0.dll
bin/cygnetcdf-11.dll -> lib/libnetcdf-11.dll
bin/cygnetcdff-6.dll -> lib/libnetcdff-6.dll
Then we can try once more to convince cmake that we have all required libraries:
cmake .. -DBUILD_NETCDF=NO -DBUILD_GRIB_API=NO -DNETCDF_DIR=auxiliary/ -DGRIB_API_DIR=auxiliary/ -DCMAKE_INSTALL_PREFIX=../_install
The problem is that on Windows we have *.dll
and *.lib
(*.dll.a
in Cygwin) standing for the actual shared object and its import library. And it seems that Cygwin places in the lib
directory only static libs and import libraries and this makes the Open-SURFEX build to go crazy.
Hi!
I'm afraid the -DNETCDF_DIR=auxiliary/ -DGRIB_API_DIR=auxiliary/
might not be the correct directory.
On both linux and cygwin windows, I have below error message when using cmake .. -DBUILD_NETCDF=NO -DBUILD_GRIB_API=NO -DNETCDF_DIR=auxiliary/ -DGRIB_API_DIR=auxiliary/ -DCMAKE_INSTALL_PREFIX=../_install
I typed that command under open-SURFEX/build.
From what I am seeing, it might be the directory is not correct---since the linux version also can't find the library.
Instead of auxiliary/
, I also tried auxiliary/bin/
and auxiliary/lib/
, neither worked.
What do you think?
Or can we simply specify the directory in cmake/FindNetCDF.cmake
? Could you possibly tell me how?
$ cmake .. -DBUILD_NETCDF=NO -DBUILD_GRIB_API=NO -DNETCDF_DIR=auxiliary/ -DGRIB_API_DIR=auxiliary/ -DCMAKE_INSTALL_PREFIX=../_install
-- Using /home/alaph/gao/open-SURFEX/conf/config.GNU.cmake
-- Failed to find NetCDF interface for F90
NetCDF_libs: NETCDF_LIBRARY-NOTFOUND
NetCDF_includes: NETCDF_INCLUDE_DIR-NOTFOUND
CMake Error at /usr/share/cmake-3.12.0-rc3/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
Could NOT find NetCDF (missing: NETCDF_HAS_INTERFACES)
Call Stack (most recent call first):
/usr/share/cmake-3.12.0-rc3/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
cmake/FindNetCDF.cmake:162 (find_package_handle_standard_args)
cmake/Dependencies.cmake:97 (find_package)
CMakeLists.txt:46 (include)
Update:
I located the problem.
We have to move auxiliay
folder elsewhere and remove all files under build
In this way the NetCDF could be successfully detected.
Great, have you moved *.ddl
from bin
to lib
to make it working, or cmake recognized *.dll.a
?
It seems that under Windows cmake expects the both import and dll parts of the library to be provided while defining an IMPORTED library. The following patch adds the missing import-parts. With this patch it should be possible to build Open-SURFEX when configured in the normal way via cmake .. -DCMAKE_INSTALL_PREFIX=../_install
, could you try it?
diff --git a/cmake/Dependencies.cmake b/cmake/Dependencies.cmake
index 83c8860..c095fe2 100644
--- a/cmake/Dependencies.cmake
+++ b/cmake/Dependencies.cmake
@@ -1,6 +1,11 @@
include(ExternalProject)
+set(shared_lib_dir "lib")
+if(CYGWIN)
+ set(shared_lib_dir "bin")
+endif()
+
function(generate_install_targets)
cmake_parse_arguments(PARSED_ARGS "" "" "EXTERNAL_PROJECTS" ${ARGN})
foreach(item ${PARSED_ARGS_EXTERNAL_PROJECTS})
@@ -76,7 +81,7 @@ if(${BUILD_NETCDF})
CMAKE_CACHE_ARGS
"-DCMAKE_Fortran_COMPILER:FILEPATH=${CMAKE_Fortran_COMPILER}"
BUILD_BYPRODUCTS
- "auxiliary/lib/libnetcdff.so"
+ "auxiliary/${shared_lib_dir}/${CMAKE_SHARED_LIBRARY_PREFIX}netcdff${CMAKE_SHARED_LIBRARY_SUFFIX}"
)
ExternalProject_get_property(NetCDF_Fortran install_dir)
@@ -88,7 +93,11 @@ if(${BUILD_NETCDF})
add_library(NetCDF::NetCDF_Fortran SHARED IMPORTED)
set_property(TARGET NetCDF::NetCDF_Fortran PROPERTY
- IMPORTED_LOCATION "${install_dir}/lib/libnetcdff.so")
+ IMPORTED_LOCATION "${install_dir}/${shared_lib_dir}/${CMAKE_SHARED_LIBRARY_PREFIX}netcdff${CMAKE_SHARED_LIBRARY_SUFFIX}")
+ if(WIN32 OR CYGWIN OR MINGW)
+ set_property(TARGET NetCDF::NetCDF_Fortran PROPERTY
+ IMPORTED_IMPLIB "${install_dir}/lib/${CMAKE_IMPORT_LIBRARY_PREFIX}netcdff${CMAKE_IMPORT_LIBRARY_SUFFIX}")
+ endif()
set_property(TARGET NetCDF::NetCDF_Fortran PROPERTY
INTERFACE_INCLUDE_DIRECTORIES "${install_dir}/include")
@@ -113,7 +122,7 @@ if(${BUILD_GRIB_API})
CMAKE_CACHE_ARGS
"-DCMAKE_Fortran_COMPILER:FILEPATH=${CMAKE_Fortran_COMPILER}"
BUILD_BYPRODUCTS
- "auxiliary/lib/libgrib_api_f90.so"
+ "auxiliary/${shared_lib_dir}/${CMAKE_SHARED_LIBRARY_PREFIX}grib_api_f90${CMAKE_SHARED_LIBRARY_SUFFIX}"
)
ExternalProject_get_property(grib_api install_dir)
@@ -125,7 +134,11 @@ if(${BUILD_GRIB_API})
add_library(grib_api::grib_api_Fortran SHARED IMPORTED)
set_property(TARGET grib_api::grib_api_Fortran PROPERTY
- IMPORTED_LOCATION "${install_dir}/lib/libgrib_api_f90.so")
+ IMPORTED_LOCATION "${install_dir}/${shared_lib_dir}/${CMAKE_SHARED_LIBRARY_PREFIX}grib_api_f90${CMAKE_SHARED_LIBRARY_SUFFIX}")
+ if(WIN32 OR CYGWIN OR MINGW)
+ set_property(TARGET grib_api::grib_api_Fortran PROPERTY
+ IMPORTED_IMPLIB "${install_dir}/lib/${CMAKE_IMPORT_LIBRARY_PREFIX}grib_api_f90${CMAKE_IMPORT_LIBRARY_SUFFIX}")
+ endif()
set_property(TARGET grib_api::grib_api_Fortran PROPERTY
INTERFACE_INCLUDE_DIRECTORIES "${install_dir}/include")
Thank you for the reply!!
Regarding the patch, adding -DDISABLE_OS_CHECK
is also necessary from what I'm seeing. Other than that, I don't have any error building SURFEX.
In addition, building result of cmake .. -DCMAKE_INSTALL_PREFIX=../_install
seems to differ from that of cmake .. -DBUILD_NETCDF=NO -DBUILD_GRIB_API=NO -DNETCDF_DIR=auxiliary/ -DGRIB_API_DIR=auxiliary/ -DCMAKE_INSTALL_PREFIX=../_install
.
In former case, under _install
there are two folders bin
and lib
. While in latter case, there is only bin
under _install
folder.
In both way of building up, I ran into an error while testing with Alp_for_0203
Using commands:
cmake .. -DCMAKE_INSTALL_PREFIX=../_install -DUSER_SOURCES_DIR=../src/FORC
cmake --build . --target pre_input_experiment
cmake --build . --target install
cd ..
cd MY_RUN/ECOCLIMAP
gfortran -fdefault-real-8 -fdefault-double-8 -g -fno-second-underscore -fpic -ffpe-trap=overflow,zero,invalid -fbacktrace -fconvert=swap convert_ecoclimap_param.f90
./a.exe
cd ..
cp -rv NAMELIST/Alp_for_0203 .
cd Alp_for_0203
cp -v ../ECOCLIMAP/*.dat .
cp -v ../ECOCLIMAP/*.bin .
../../_install/bin/PRE_INPUT_EXPERIMENT.exe
The error message is:
$ ../../_install/bin/PRE_INPUT_EXPERIMENT.exe
C:/cygwin64/home/alaph/gao/open-SURFEX/_install/bin/PRE_INPUT_EXPERIMENT.exe: error while loading shared libraries: cygnetcdff-6.dll: cannot open shared object file: No such file or directory
What do you think?
Thank you very much!
Yes, of course, I forgot about that -DDISABLE_OS_CHECK
!
As for the different directory structure, it is expected behavior. When you configure Open-SURFEX with cmake .. -DCMAKE_INSTALL_PREFIX=../_install
you basically asking to build some required libraries along with the main Open-SURFEX code, thats why you have lib
directory after install. Contrary, when configuring with cmake .. -DBUILD_NETCDF=NO -DBUILD_GRIB_API=NO -DNETCDF_DIR=auxiliary/
you tell that you already have required libraries installed, so you'll get just a bin
directory with Open-SURFEX binaries.
And from the latest error message it seems that dll's locations aren't hardcoded in built executables. So, I think that the bin
directory under _install/lib/surfex/bin
should be added to the PATH environment variable. Or just copy these *.dll
to be in the same directory as Open-SURFEX binaries.
BTW, now it is possible to provide multiple user directories. So by using
cmake .. -DCMAKE_INSTALL_PREFIX=../_install -DUSER_SOURCES_DIR='../src/FORC;../MY_RUN/ECOCLIMAP'
you'll get CONVERT_ECOCLIMAP_PARAM
in the bin
directory as well as the rest of Open-SURFEX binaries, hopefully.
Thank you very much!
I copied libraries whose names start with cyg*
under _install/lib/surfex/bin
to MY_RUN/Alp_for_0203
and it did solve my problem!
I don't really understand what are treated as user directories
.
I thought ECOCLIMAP
is a dataset so it should be a user directory.
But what does src/FORC
mean?
Could you possibly suggest some readings towards what these folders mean?
In terms of build system, user directories
are additional directories with source files that could be merged with the main source tree during the build.
Yes, that's true ECOCLIMAP
directory contains some datasets, but a Fortran program to convert these datasets from ASCII to binary is also placed in this directory. So, you had to compile it manually:
gfortran -fdefault-real-8 -fdefault-double-8 -g -fno-second-underscore -fpic -ffpe-trap=overflow,zero,invalid -fbacktrace -fconvert=swap convert_ecoclimap_param.f90
But if you pass it as a user directory
, that source file would be merged into the main source tree and built along with Open-SURFEX binaries.
For additional information, I'd recommend to consult the SURFEX User Guide.
Thank you very much!
Regarding convert_ecoclimap_param
, perhaps I shall raise questions in issue #4
Thank you very much! I wrote below steps to step up open-SURFEX on windows. I hope someone else could possibly benefit from it in the future and possibly point out what is not correct.
Select Packages
, select view by Category
and also single click Devel
in order that the entry status changes from default
to install
# download source
wget https://cmake.org/files/v3.12/cmake-3.12.0.zip
# download binary
wget https://cmake.org/files/v3.12/cmake-3.12.0-win64-x64.zip
unzip cmake-3.12.0.zip
unzip cmake-3.12.0-win64-x64.zip
cp -r cmake-3.12.0-win64-x64/share/cmake-3.12 /usr/share
cd cmake-3.12.0
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=../_install
cmake --build .
cmake --build . --target install
cd ../_install/bin
rm /usr/bin/cmake.exe
ln `pwd`/cmake.exe /usr/bin/cmake.exe
git clone https://github.com/joewkr/open-SURFEX/
cd open-SURFEX
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=../_install
cmake --build .
cmake --build . --target install
open-SURFEX/_install/bin/
. If you build cygwin under default configuration, you could find open-SURFEX
under C:\cygwin64\home\
ECOCLIMAP
cd ~/open-SURFEX/build/
cmake .. -DCMAKE_INSTALL_PREFIX=../_install -DUSER_SOURCES_DIR=../src/FORC
cmake --build . --target pre_input_experiment
cmake --build . --target install
cd ~/open-SURFEX/MY_RUN/ECOCLIMAP
gfortran -fdefault-real-8 -fdefault-double-8 -g -fno-second-underscore -fpic -ffpe-trap=overflow,zero,invalid -fbacktrace -fconvert=swap convert_ecoclimap_param.f90
./a.exe
# in dataset the data is encoded in ascii
# we are converting data to binary form
# In this example we are using Alp_for_0203 data in ECOCLIMAP
cd ..
cp -r NAMELIST/Alp_for_0203 .
cd Alp_for_0203
cp ../ECOCLIMAP/*.dat .
cp ../ECOCLIMAP/*.bin .
# Then we have to copy the library to where we execute our commands
cp ~/open-SURFEX/_install/lib/surfex/bin/cyg* .
# We are executing the binaries under ~/open-SURFEX/MY_RUN/ECOCLIMAP/Alp_for_0203
../../_install/bin/PRE_INPUT_EXPERIMENT
# PGD -- to prepare physiography data
# PREP -- to prepare initial state of an experiment
# OFFLINE -- to run an experiment
../../_install/bin/PGD
../../_install/bin/PREP
../../_install/bin/OFFLINE
Nice step-by-step guide, I'd say! BTW, which cmake version would you get with cygwin?
Thank you! I forgot to mention how I set up cmake on cygwin! I am using CMake 3.12.0-r3. I built cmake from source and added a hard link to /usr/bin/cmake.exe Then I copied share/cmake-3.12.0 from the binary distribution to /usr/share . I didn't know why but adding a hard link from the cmake.exe in binary distribution to /usr/bin/cmake.exe will cause error. Copying cmake.exe will also cause error.
Below is how I built cmake-3.12.0
# download source
wget https://cmake.org/files/v3.12/cmake-3.12.0.zip
# download binary
wget https://cmake.org/files/v3.12/cmake-3.12.0-win64-x64.zip
unzip cmake-3.12.0.zip
unzip cmake-3.12.0-win64-x64.zip
cp -r cmake-3.12.0-win64-x64/share/cmake-3.12 /usr/share
cd cmake-3.12.0
mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=../_install
cmake --build .
cmake --build . --target install
cd ../_install/bin
rm /usr/bin/cmake.exe
ln `pwd`/cmake.exe /usr/bin/cmake.exe
Hi!
I am building open-SURFEX on windows with Cygwin. I am using CMake version 3.12.0. I have successfully built open-SURFEX on my debian before so I was using the same commands on cygwin.
When I type
mkdir build && cd build
,cmake .. -DCMAKE_INSTALL_PREFIX=../_install
, the error message below shows up.The problems seem to be that CMake thinks there are already targets with the same name. But I used
find -name 'prep'
and I did not find any targets with the same name.This is the log: CMakeOutput.log
Why this is happening? Could you suggest a possible solution for this?
Thank you in advance!