search

CABLE-LSM / benchcab

Tool for evaluation of CABLE land surface model
https://benchcab.readthedocs.io/en/latest/
Apache License 2.0
2 stars 4 forks source link

Should we parallelise the compilations across the versions? #130

Closed ccarouge closed 6 months ago

ccarouge commented 1 year ago

The compilation takes some time. It should be possible to reduce that time by using parallel make (make -j 4) by almost a factor of 4. But should we also consider parallelising the compilation across the model versions?

I am also wondering, if when we will run both the fluxsite tests and spatial tests, we should consider doing all the compilations at once in parallel: all code versions and serial and MPI compilations. We could have 2 copies of the source code, one for serial compilation and one for MPI?

SeanBryan51 commented 1 year ago

We could have 2 copies of the source code, one for serial compilation and one for MPI?

I would like to avoid having duplicate copies of the same repository on the file system if possible. An alternative could be to have two separate .tmp directories: one for serial and one for MPI (this was actually the case in the CABLE-2.0 build script). This way the serial build does not interfere with the MPI build. It also saves us from rebuilding cable executables whenever we want to switch from a serial build to an MPI build every time.

SeanBryan51 commented 1 year ago

I've been having some more problems with build script: see https://github.com/CABLE-LSM/benchcab/pull/136#issuecomment-1715137375. Should we consider working directly with the Makefile instead of the build script?

ccarouge commented 1 year ago

If using the build script is hampering us, then let's ditch it for benchcab. Happy to go with a different .tmp directory instead of a full copy of the code. The .tmp directory is already a copy...

SeanBryan51 commented 6 months ago

Building the MPI component of CABLE is now built along side the serial component due to the recent CMake changes to CABLE. Separate builds for serial and MPI applications are no longer necessary.

ccarouge commented 6 months ago

Considering for the moment, we don't have use cases that require more than 2 model versions, we don't need to parallelise the compilation more than that. I'll close this issue for now.