Gaea C5 lib issue - Githubissues

ufs-community / ufs-weather-model

UFS Weather Model

Other

142 stars 249 forks source link

Gaea C5 lib issue #2472

Closed jkbk2004 closed 1 month ago

jkbk2004 commented 1 month ago

Description

Gaea C5 system has been updated during the week of Oct 15 2024: compilers, mpi packges, etc.

Spack stack maintenance should be followed up.

ERROR: ld.so: object '/opt/cray/pe/gcc/12.2.0/snos/lib64/libstdc++.so.6' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
Lmod has detected the following error: The following module(s) are unknown: "intel-classic/2023.1.0"

To Reproduce:

Additional context

Output

jkbk2004 commented 1 month ago

@ulmononian @RatkoVasic-NOAA this is highest priority on the library team.

JustinPerket commented 1 month ago

I'm not sure if it's helpful, but additionally there's no longer a "PrgEnv-intel/8.3.3" module used in rt.sh, but now C5 has "PrgEnv-intel/8.5.0". Resulting in this error:

rt.sh: Setting up gaea...
ERROR: ld.so: object '/opt/cray/pe/gcc/12.2.0/snos/lib64/libstdc++.so.6' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
Lmod has detected the following error:  The following module(s) are unknown: "PrgEnv-intel/8.3.3"

RatkoVasic-NOAA commented 1 month ago

@jkbk2004 I will reinstall spack-stack-1.6.0, please don't use Gaea-c5 as I will rename old installation and it is not going to work until finished. It can take up to one day. I will let you know when it is done.

RatkoVasic-NOAA commented 1 month ago

Spack stack on Gaea C5 is finished. I kept same path/name: /ncrc/proj/epic/spack-stack/spack-stack-1.6.0/envs/fms-2024.01/install/modulefiles/Core Changes are needed in rt.sh part:

    export LD_PRELOAD=/usr/lib64/libstdc++.so.6
    module load PrgEnv-intel/8.5.0
    module load intel-classic/2023.2.0
    module load cray-mpich/8.1.28
    module load python/3.9.12

NOTE! still not tested what else should be changed in rt.sh

jkbk2004 commented 1 month ago

@RatkoVasic-NOAA intel-classic-mixed/2023.2.0 or intel-mixed/2023.2.0 ? I don't see intel-classic/2023.2.0

RatkoVasic-NOAA commented 1 month ago

[role.epic@gaea56:/ncrc/proj/epic/spack-stack/spack-stack-1.6.0/envs/unified-env]# module list

Currently Loaded Modules:
  1) craype-x86-rome          5) xpmem/2.9.6-1.1_20240510205610__g087dc11fc19d   9) PrgEnv-intel/8.5.0     13) TimeZoneEDT/default     17) cray-mpich/8.1.28
  2) libfabric/1.20.1         6) craype/2.7.30                                  10) cray-pmi/6.1.13        14) DefApps/default         18) python/3.9.12
  3) craype-network-ofi       7) cray-dsmml/0.2.2                               11) darshan-runtime/3.4.0  15) git-lfs/2.11.0          19) qt/5.15.2
  4) perftools-base/23.12.0   8) cray-libsci/24.07.0                            12) CmrsEnv/default        16) intel-classic/2023.2.0  20) ecflow/5.8.4

RatkoVasic-NOAA commented 1 month ago

Also change in ./modulefiles/ufs_gaea.intel.lua:

stack_intel_ver=os.getenv("stack_intel_ver") or "2023.2.0"
load(pathJoin("stack-intel", stack_intel_ver))

stack_cray_mpich_ver=os.getenv("stack_cray_mpich_ver") or "8.1.28"
load(pathJoin("stack-cray-mpich", stack_cray_mpich_ver))

RatkoVasic-NOAA commented 1 month ago

Another change in ./tests/run_test.sh:

-    module load stack-intel/2023.1.0 stack-cray-mpich/8.1.25
+    module load stack-intel/2023.2.0 stack-cray-mpich/8.1.28

RatkoVasic-NOAA commented 1 month ago

https://github.com/ufs-community/ufs-weather-model/pull/2448#issuecomment-2421231057

jkbk2004 commented 1 month ago

Thanks @RatkoVasic-NOAA ! I pushed gaea-c5 update to #2449