aamaricci / SciFortran

A library of fortran modules and routines for scientific calculations (*in a way* just like scipy for python)
GNU Lesser General Public License v3.0
160 stars 39 forks source link

We should better define the min versions of `CMake` and `ninja` #16

Closed beddalumia closed 1 year ago

beddalumia commented 1 year ago

Getting ninja into HPC environments means either to deal with manual binary download (or build) or to rely on nonsudo solutions.

From their official list I think we could choose among these options:

E.g. on Ulysses, as far as I know, we have only pip out of the box, but it bundles a slightly outdated version of Ninja, which will not work for SciFortran. Here what happens:

$ cmake -GNinja ..
CMake Error at CMakeLists.txt:5 (PROJECT):
  The Ninja generator does not support Fortran using Ninja version

    1.10.2.git.kitware.jobserver-1

  due to lack of required features.  Kitware has implemented the required
  features but as of this version of CMake they have not been integrated to
  upstream ninja.  Pending integration, Kitware maintains a branch at:

    https://github.com/Kitware/ninja/tree/features-for-fortran#readme

  with the required features.  One may build ninja from that branch to get
  support for Fortran.

I managed to easily try the latest brewed version (1.11.1) on my laptop and everything works fine. But HomeBrew is quite of an hassle to get on Ulysses (and elsewhere I suspect) since it complains about a too old version of curl (and who knows what else downstream...).

Summarizing for now we know that: CMake Version / Ninja Version Does work?
v3.15.4 / v1.10.2
vx.yy.z / v1.11.0
v3.24.2 / v1.11.1 🟢

Ninja version 1.11.0 apparently is shipped by both Conda and Spack, which even has a separate package aimed at supporting fortran, so we could try one of those to determine the minimum required version for Ninja to work here. In the end we should edit the README to redirect to a fine-tuned subset of eligible Ninja versions.

beddalumia commented 1 year ago

I'll start trying with (mini)conda, which should be easy to install following this: https://github.com/um-dang/conda_on_the_cluster

beddalumia commented 1 year ago

No, apparently the problem is the version of CMake:

(base) cmake -GNinja ..
CMake Error at CMakeLists.txt:5 (PROJECT):
  The Ninja generator does not support Fortran using Ninja version

    1.11.0

  due to lack of required features.  Kitware has implemented the required
  features but as of this version of CMake they have not been integrated to
  upstream ninja.  Pending integration, Kitware maintains a branch at:

    https://github.com/Kitware/ninja/tree/features-for-fortran#readme

  with the required features.  One may build ninja from that branch to get
  support for Fortran.

(base) cmake --version
cmake version 3.15.4

CMake suite maintained and supported by Kitware (kitware.com/cmake).

(and the CMake error messages quite misleading...)

beddalumia commented 1 year ago

If one needs finer control on the CMake version to have the -GNinja option to work it can become a though problem to deal on clusters... on Ulysses we just have:

$ module spider CMake

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
  cmake: cmake/3.15.4
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Description:
      CMake is an open-source, cross-platform family of tools designed to build, test and package software. 

    This module can be loaded directly: module load cmake/3.15.4

    Help:

      This module loads the cmake utility

      Version 3.15.4

It's a bit of a pity since Ninja is really way faster then make and I wanted that on Ulysses 🥲

beddalumia commented 1 year ago

Using Conda's version of CMake I managed to build SciFortran with Ninja. Good.

Still I believe we should try to find out what's the minimum version for CMake that can achieve this and state it in the README. Finding a non-conda way to get it would also be much better imho (never really liked the Conda lands and this brief trip therein has not improved my feeling in the slightest 🙃 )

beddalumia commented 1 year ago

Definitive statement

Comparing this:

https://cmake.org/cmake/help/v3.16/generator/Ninja.html#fortran-support

to this:

https://cmake.org/cmake/help/v3.17/generator/Ninja.html#fortran-support

One would guess that CMake was not able to exploit the Fortran support that Ninja>=1.10 provides, until version 3.16, but from 3.17 onwards yes. Trusting this information we could state that the Ninja build requires CMake >= 3.17 and Ninja>=1.10 and that's it.

Though I decided to give it a try (via Spack) and, somewhat surprisingly, found that both CMake v3.16.9 and v3.16.8 can indeed generate functional ninja files, with Ninja v1.10.2. Despite Spack comes across as a thing of beauty[^1], installing many many versions of CMake just to determine the minimum is very time consuming and I do not intend to keep doing this right now: @aamaricci, what if we just state that the combination of CMake=3.17 and Ninja=1.10.2 is known to work fine and that earlier versions are not guaranteed to?

Ninja>=1.10.2 would not be a hard requirement, since it is very easily installable through pip. CMake can be a little more of a hassle on HPC environments, but we could just suggest to look at Spack and that's it, afterall there is always make...

...speaking of which: of course the softer CMake > 3.0.0 requirement can be kept, specifying that it allows make-based builds only.

[^1]: it even has full support for Environment Modules!

aamaricci commented 1 year ago

Totally fine for me. Thanks a lot Gabriele for fixing this. When I tested Ninja I didn't actually checked for the version.

We can update che CMakeLists.txt file accordingly and write a note in the Readme.

beddalumia commented 1 year ago

Small update, just for reference: CMake v3.16.3 (what apt currently ships on Ubuntu 20.04 LTS) does work with Ninja v1.11.1 I am updating the README right now: up in the dependency list I will just put a vague CMake > 1.16 and Ninja > 1.10 and defer to a footnote for a brief discussion and a link to this issue in the form of 'for more information see...'.

I would avoid bumping the CMAKE_MINIMUM_REQUIRED parameter as CMake ≥ 3.0 works perfectly with make and I guess some people would want that to keep working, without any action on their part.

aamaricci commented 1 year ago

Perfect. I do agree.