PISM not compiling on mistral

ackerlar commented 3 years ago

Describe the bug I know this already has been an issue but I could not find a working solution for me. When I try to compile pism-github1.2.1 on mistral, I get the following error:

When I run esm_master comp-pism-github1.2.1 using esm_tools on release:

[ 16%] Building CXX object src/coupler/CMakeFiles/boundary.dir/frontalmelt/Constant.cc.o
cd /pf/a/a270124/pism-github1.2.1/build/src/coupler && /sw/rhel6-x64/intel/impi/2018.5.288/compilers_and_libraries/linux/mpi/bin64/mpiicpc   -I/sw/rhel6-x64/netcdf/netcdf_c-4.3.2-parallel-impi-intel14/include -I/sw/rhel6-x64/util/udunits-2.2.26-gcc64/include -I/sw/rhel6-x64/numerics/fftw-3.3.7-openmp-gcc64/include -I/sw/rhel6-x64/numerics/PETSc-3.12.2-impi2018-intel18/include -I/pf/a/a270124/pism-github1.2.1/build  -std=c++11 -O3 -fp-model precise -no-fast-transcendentals -march=native -O2 -g -DNDEBUG   -fPIC -std=c++11 -o CMakeFiles/boundary.dir/frontalmelt/Constant.cc.o -c /pf/a/a270124/pism-github1.2.1/src/coupler/frontalmelt/Constant.cc
/pf/a/a270124/pism-github1.2.1/src/coupler/frontalmelt/FrontalMelt.cc(126): error: no instance of constructor "pism::frontalmelt::FrontalMelt::FrontalMelt" matches the argument list
            argument types are: (pism::IceGrid::ConstPtr, std::nullptr_t)
    : FrontalMelt(g, nullptr) {
                 ^
/pf/a/a270124/pism-github1.2.1/build/pism/coupler/FrontalMelt.hh(46): note: this candidate was rejected because mismatch in count of arguments
  class FrontalMelt : public Component {
        ^
/pf/a/a270124/pism-github1.2.1/src/coupler/frontalmelt/FrontalMelt.cc(125): note: this candidate was rejected because mismatch in count of arguments
  FrontalMelt::FrontalMelt(IceGrid::ConstPtr g)
               ^
/pf/a/a270124/pism-github1.2.1/src/coupler/frontalmelt/FrontalMelt.cc(114): note: this candidate was rejected because arguments do not match
  FrontalMelt::FrontalMelt(IceGrid::ConstPtr g, std::shared_ptr<FrontalMelt> input)
               ^

compilation aborted for /pf/a/a270124/pism-github1.2.1/src/coupler/frontalmelt/FrontalMelt.cc (code 2)
make[2]: *** [src/coupler/CMakeFiles/boundary.dir/frontalmelt/FrontalMelt.cc.o] Error 2
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory `/mnt/lustre01/pf/a/a270124/pism-github1.2.1/build'
make[1]: *** [src/coupler/CMakeFiles/boundary.dir/all] Error 2
make[1]: Leaving directory `/mnt/lustre01/pf/a/a270124/pism-github1.2.1/build'
make: *** [all] Error 2
Traceback (most recent call last):
  File "/pf/a/a270124/.local/bin/esm_master", line 8, in <module>
    sys.exit(main())
  File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_master/cli.py", line 103, in main
    main_flow(parsed_args, target)
  File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_master/esm_master.py", line 84, in main_flow
    user_task.execute(ignore_errors) #env)
  File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_master/task.py", line 492, in execute
    shell=(command.startswith("./") and command.endswith(".sh")),
  File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['./comp-pism-github1.2.1_script.sh']' returned non-zero exit status 2.

When I apply the changes from hotfix/mistral_pism branch, I get this error:

[ 32%] Building CXX object src/coupler/CMakeFiles/boundary.dir/surface/EISMINTII.cc.o
cd /pf/a/a270124/pism-github1.2.1/build/src/coupler && /sw/rhel6-x64/intel/impi/2018.5.288/compilers_and_libraries/linux/mpi/bin64/mpiicpc   -I/sw/rhel6-x64/netcdf/netcdf_c-4.4.1.1-parallel-impi-intel14/include -I/sw/rhel6-x64/util/udunits-2.2.26-gcc64/include -I/sw/rhel6-x64/numerics/fftw-3.3.7-openmp-gcc64/include -I/sw/rhel6-x64/numerics/PETSc-3.12.2-impi2018-intel18/include -I/pf/a/a270124/pism-github1.2.1/build  -std=c++11 -O3 -fp-model precise -no-fast-transcendentals -march=native -gcc-name=/sw/rhel6-x64/gcc/gcc-6.4.0/bin/gcc -gxx-name=/sw/rhel6-x64/gcc/gcc-6.4.0/bin/g++ -O2 -g -DNDEBUG   -fPIC -std=c++11 -o CMakeFiles/boundary.dir/surface/EISMINTII.cc.o -c /pf/a/a270124/pism-github1.2.1/src/coupler/surface/EISMINTII.cc
make[2]: Leaving directory `/mnt/lustre01/pf/a/a270124/pism-github1.2.1/build'
[ 32%] Built target boundary
make[1]: Leaving directory `/mnt/lustre01/pf/a/a270124/pism-github1.2.1/build'
make: *** [all] Error 2
Traceback (most recent call last):
  File "/pf/a/a270124/.local/bin/esm_master", line 8, in <module>
    sys.exit(main())
  File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_master/cli.py", line 103, in main
    main_flow(parsed_args, target)
  File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_master/esm_master.py", line 84, in main_flow
    user_task.execute(ignore_errors) #env)
  File "/pf/a/a270124/.local/lib/python3.6/site-packages/esm_master/task.py", line 492, in execute
    shell=(command.startswith("./") and command.endswith(".sh")),
  File "/sw/rhel6-x64/conda/anaconda3-bleeding_edge/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['./comp-pism-github1.2.1_script.sh']' returned non-zero exit status 2.

System (please complete the following information):

Supercomputer: mistral

Version:

+---------------------+-------------+-----------------------------------------------------------------------+----------+---------+
| package_name        | version     | file                                                                  | branch   | tags    |
|---------------------+-------------+-----------------------------------------------------------------------+----------+---------|
| esm_analysis        | 0.4.2       | /mnt/lustre01/pf/a/a270124/esm_analysis                               |          |         |
| esm_calendar        | 5.0.0       | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
| esm_database        | 5.0.0       | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
| esm_environment     | 5.1.1       | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
| esm_master          | 5.1.3       | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
| esm_motd            | 5.0.2       | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
| esm_parser          | 5.1.7       | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
| esm_pism            | 0.0.1.dev15 | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
| esm_plugin_manager  | 5.0.1       | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
| esm_profile         | 5.0.0       | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
| esm_project         | 0.1.0       | /mnt/lustre02/work/ba0989/a270124/processor-partitioning/python_tools |          |         |
| esm_rcfile          | 5.1.0       | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
| esm_runscripts      | 5.1.25      | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
| esm_tools           | 5.1.17      | /mnt/lustre01/pf/a/a270124/esm_tools                                  | release  | v5.1.17 |
| esm_version_checker | 5.1.4       | /mnt/lustre01/pf/a/a270124/.local/lib/python3.6/site-packages         |          |         |
+---------------------+-------------+-----------------------------------------------------------------------+----------+---------+

pgierz commented 3 years ago

To refresh my memory, the hot fix pism branch was mine I did sometime in spring, right?

I'll have another look at the build tomorrow. In the very worst case, we just build pism somehow by hand and bind it in. Although that might induce runtime incompatibilities. I'll put it on my todo list.

ackerlar commented 3 years ago

@pgierz Yes, here is the pull request #276

ackerlar commented 3 years ago

I managed to compile a running PISM with the preinstalled dependencies. However, I used my own script (/pf/a/a270124/pism_build_sebastian02/BuildPismMistral.sh) for compiling and I had to use a different gcc version in configs/machines/mistral.yaml:

165 LD_LIBRARY_PATH[(1)]: /sw/spack-rhel6/gcc-9.1.0-lsnsye/lib64:$LD_LIBRARY_PATH

and

165 LD_LIBRARY_PATH[(1)]: /sw/rhel6-x64/gcc/gcc-5.1.0/lib64:$LD_LIBRARY_PATH

seem to work.

I am trying to include the building settings into esm_tools

denizural commented 3 years ago

Hi @ackerlar, when you build PISM can you also redirect the output with &>. Sometimes models fully compile but there are warning messages which say that there can be runtime issues.

pgierz commented 3 years ago

Alright, my own two cents: I am also able to build a PISM reliably one mistral, but with self-made scripts. @ackerlar, it sounds like you and I are currently in the middle of doing the same thing.

I would recommend the following:

Lars keeps testing with the self-made variant to get that to run
We build a "pism" module, which will live in one of the paleoclimate group folders (probably /work/ab0246/pool or /work/ab0246/models, which doesn't exist yet, but I'll talk to Christian and get that to happen)
We tell esm-tools to just load and use that module, which has all the right versions and dependencies (I can take care of this part)
Self-built pism directly through esm-tools becomes forbidden unless you know exactly what you are doing, there are too many idiotic things that can happen.

Thoughts? Comments?

ackerlar commented 3 years ago

@denizural a log file is here /pf/a/a270124/pism_build_sebastian02/build_pism.48828.log. Except for:

CMake Warning:
  Manually-specified variables were not used by the project:

    Pism_USE_PARALLEL_HDF5

I cannot find any warning. However, I guess this warning shouldn't occur?

@pgierz sounds good! I will do some test runs and keep you updated.

ackerlar commented 2 years ago

solved in #547

esm-tools / esm_tools

PISM not compiling on mistral #413