search

esmf-org / esmf

The Earth System Modeling Framework (ESMF) is a suite of software tools for developing high-performance, multi-component Earth science modeling applications.
https://earthsystemmodeling.org/
Other
153 stars 74 forks source link

Build config fixes for Cygwin #136

Open robin-deltares opened 1 year ago

robin-deltares commented 1 year ago

At Deltares we use an old version of ESMF on Windows and I'm trying to build a recent version.

Using the MinGW build config works, but that is i686-based.

Copying some MinGW flags to the Cygwin build config gets me further, but there are some non-functional (ESMF_OS_MinGW instead of ESMF_NO_SIGNALS, for instance) code-switches that still make it fail.

I would like to request permission to create a feature-branch for correcting the Cygwin flags and making the code-switches functional.

theurich commented 1 year ago

@robin-deltares best just to make fork, create a branch under your fork, and then issue a PR when ready to be reviewed and merged back into the ESMF develop branch (or maybe even a patch branch for a recent release). Let us know if you have questions. Thank you for contributing!

robin-deltares commented 1 year ago

I've progressed a lot.

Changes:

  1. Moved over the Windows-specific build config flags from MinGW to Cygwin.
  2. Changed the ESMF_OS_MinGW to OSMF_NO_SIGNALS (simply because they share this setting and I don't know what would be appropriate for each code-block) in the code in a hand full of files.
  3. /usr/include/w32api/Winsock.h uses different formats for time, e.g. error: cannot convert ‘long unsigned int*’ to ‘__ms_u_long*’ {aka ‘unsigned int*’} Bypassed it with ESMF_NO_SOCKETS for now.

When I've reached a successful build, I'd like to have working code put upstream in a more appropriate manner so that we can build future releases straight from the source, but then I'd need some help.

robin-deltares commented 1 year ago

Infrastructure and Superstructure were able to build.

Had some problems with scopes though, e.g.:

src/Infrastructure/Mesh/src/Moab/FileOptions.cpp:58:17: error: ‘strdup’ was not declared in this scope; did you mean ‘strcmp’?

src/Infrastructure/Mesh/src/Moab/GeomQueryTool.cpp:1495:27: error: ‘M_PI’ was not declared in this scope

Will now try to build the whole package.

robin-deltares commented 1 year ago

🎉 The suite has been built.

# !!! The following options were used on this ESMF build !!!
#
# ESMF_DIR: /home/westrene/git/esmf
# ESMF_OS: Cygwin
# ESMF_MACHINE: x86_64
# ESMF_ABI: 64
# ESMF_COMPILER: gfortran
# ESMF_BOPT: O
# ESMF_COMM: openmpi
# ESMF_SITE: default
# ESMF_PTHREADS: OFF
# ESMF_OPENMP: ON
# ESMF_OPENACC: OFF
# ESMF_ARRAY_LITE: FALSE
# ESMF_NO_INTEGER_1_BYTE: TRUE
# ESMF_NO_INTEGER_2_BYTE: TRUE
# ESMF_FORTRANSYMBOLS: default
# ESMF_MAPPER_BUILD: OFF
# ESMF_AUTO_LIB_BUILD: ON
# ESMF_DEFER_LIB_BUILD: ON
# ESMF_SHARED_LIB_BUILD: ON
#
# ESMF environment variables pointing to 3rd party software:
# ESMF_MOAB:              internal
# ESMF_LAPACK:            internal
# ESMF_ACC_SOFTWARE_STACK:            none
# ESMF_NETCDF:            nc-config
# ESMF_NETCDF_INCLUDE:    /usr/include
# ESMF_NETCDF_LIBS:       -lnetcdf
# ESMF_NETCDF_LIBPATH:    /usr/lib
# ESMF_NFCONFIG:          nf-config
# ESMF_NETCDFF_INCLUDE:   /usr/include
# ESMF_NETCDFF_LIBS:      -lnetcdff -lnetcdf -lnetcdf -lm
# ESMF_NETCDFF_LIBPATH:   /usr/lib
# ESMF_YAMLCPP:           internal
#
# * Compilers, Linkers, Flags, and Libraries *
# Location of the preprocessor:      /usr/bin/gcc
# Location of the Fortran compiler:  /usr/bin/mpifort
# Location of the Fortran linker:    /usr/bin/mpifort
# Location of the C++ compiler:      /usr/bin/mpicxx
# Location of the C++ linker:        /usr/bin/mpicxx
# Location of the C compiler:        /usr/bin/mpicc
# Location of the C linker:          /usr/bin/mpicc

Building the tests now.

robin-deltares commented 1 year ago

Worked inside of Cygwin, but outside of Cygwin openmpi gives problems with missing /dev/shm and some text-file using a Unix-path.

Retrying build with mpiuni.

robin-deltares commented 1 year ago

Success.

PS C:\Users\westrene\Documents\esmf_bin> .\ESMF_RegridWeightGen.exe --version
   ESMF_VERSION_STRING:       8.5.0 beta snapshot
   ESMF_VERSION_STRING_GIT:   (not available)
   ESMF_VERSION_MAJOR:                   8
   ESMF_VERSION_MINOR:                   5
   ESMF_VERSION_REVISION:                0
   ESMF_VERSION_PATCHLEVEL:              0
   ESMF_VERSION_PUBLIC:        F
   ESMF_VERSION_BETASNAPSHOT:  T

I'm not proud of my fork, but it works on my machine.

https://github.com/esmf-org/esmf/compare/develop...robin-deltares:esmf:Fix-Cygwin-build-configuration-and-code-switches

theurich commented 1 year ago

Great progress. Please open a PR when you are ready to bring this into the mainline development branch so we can review. Thanks!

robin-deltares commented 1 year ago

Would you like me to create a PR as-is? As I think it would be most efficient to make it proper with the help of somebody who has a better understanding of the ESMF code and programming in general.

I'd be open to a Teams-meeting, for instance, to shortly explain what I changed and why, to discuss a better implementation.

theurich commented 1 year ago

I see you have a link to your fork in your earlier comment. Somehow I missed that earlier. This is fine, and no PR needed. We can take it from here.... but might ask you to test a snapshot once we have it on your machine... since the group currently does not have access to a Cygwin setup.

robin-deltares commented 1 year ago

Yes, will do!

robin-deltares commented 1 year ago

Our company-testbench failed with my build, because it needed PIO (which I disabled previously due to errors).

(I only did the ESMF-provided ESMF_RegridWeightGenUTest.exe unit test for the function earlier, which passed)

20230426 164311.432 ERROR            PET0 /home/westrene/git/esmf/src/Infrastructure/Mesh/src/ESMCI_Mesh_FileIO.C:298 ESMCI_mesh_create_from_file() Library needed by ESMF not present  - This functionality requires ESMF to be built with the PIO library enabled.
20230426 164311.432 ERROR            PET0 ESMCI_MeshCap.C:2600 MeshCap::meshcreatefromfilenew() Library needed by ESMF not present  - Internal subroutine call returned Error
20230426 164311.432 ERROR            PET0 ESMF_Mesh.F90:1969 ESMF_MeshCreateFromFile() Library needed by ESMF not present  - Internal subroutine call returned Error
20230426 164311.432 ERROR            PET0 ESMF_RegridWeightGen.F90:1278 ESMF_RegridWeightGenFile Library needed by ESMF not present  - Internal subroutine call returned Error

With some tinkering I was able to build IO with PIO enabled: https://github.com/robin-deltares/esmf/commit/4b459fdeb8d96c70c8c478555b594ab39cb51ecc

I added comments with my reasoning behind the changes.

Trying a full build now.

robin-deltares commented 1 year ago

Making the DLL failed. Mostly MPI signals that seem to not be defined for the mpiuni stub?

https://pastebin.com/FcKNmhrx

theurich commented 1 year ago

ESMF_PIO=internal is not currently supported for ESMF_COMM=mpiuni.

robin-deltares commented 1 year ago

It seems PIO is a requirement for ESMF_RegridWeightGen.exe and I was able to build and run ESMF with OpenMPI within Cygwin, but not outside of Cygwin due to /dev/shm and other Unix-paths.

What combination of options would be the way forward for me?

theurich commented 1 year ago

I suspect that the /dev/shm issue stems from MPI-3-style shared memory support we use inside ESMF. It's only for some very specific functionality, and can be disabled by setting 

ESMF_F90COMPILECPPFLAGS+= -DESMF_NO_MPI3
ESMF_CXXCOMPILECPPFLAGS+= -DESMF_NO_MPI3

in the specific (OpenMPI) section in your build_rules.mk.

theurich commented 1 year ago

@robin-deltares - we are nearing the 8.5.0 freeze, and I am looking at bringing your Cygwin port into develop before the freeze. Does your https://github.com/robin-deltares/esmf/commits/Fix-Cygwin-build-configuration-and-code-switches branch have everything you needed to make it work for you? If so, I will use it as a starting point. Thanks.

theurich commented 1 year ago

@robin-deltares, please let me know if your https://github.com/robin-deltares/esmf/commits/Fix-Cygwin-build-configuration-and-code-switches branch is ready. I'd like to avoid looking at it if it's not quite ready. Thanks!

theurich commented 1 year ago

I opened PR #149 to review the changes on @robin-deltares' branch, and bring them into a branch (https://github.com/esmf-org/esmf/tree/fix/cygwin) under the esmf-org/esmf repo. From there it can be run with full regression testing across platforms, and worked on before opening the final PR into develop. Due to the amount of changes, especially affecting source files, I expect this to be a longer process. Also we need a Cygwin test machine where changes can be tested.