Closed AlexanderRichert-NOAA closed 1 year ago
This is approximately what the output should look like (note the [^] for upstream packages):
==> Concretized ufs-weather-model-env%intel ^parallelio@2.5.10
- see3rjc ufs-weather-model-env@1.0.0%intel@19.1.3.304~debug+python build_system=bundle arch=linux-sles15-zen2
[^] rlfvsu3 ^bacio@2.4.1%intel@19.1.3.304~ipo+pic build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] padsq4n ^cmake@3.23.1%intel@19.1.3.304~doc+ncurses+ownlibs~qt build_system=generic build_type=Release arch=linux-sles15-zen2
[^] r7drzq6 ^ncurses@6.4%intel@19.1.3.304~symlinks+termlib abi=none build_system=autotools arch=linux-sles15-zen2
[^] hqthlth ^openssl@1.1.1t%intel@19.1.3.304~docs+shared build_system=generic certs=mozilla arch=linux-sles15-zen2
[^] hdfx2bu ^ca-certificates-mozilla@2023-01-10%intel@19.1.3.304 build_system=generic arch=linux-sles15-zen2
[^] 2lfnkwl ^gmake@4.4.1%intel@19.1.3.304~guile build_system=autotools arch=linux-sles15-zen2
- mbchtbo ^base-env@1.0.0%intel@19.1.3.304 build_system=bundle arch=linux-sles15-zen2
[^] 6keecbo ^curl@8.0.1%intel@19.1.3.304~gssapi~ldap~libidn2~librtmp~libssh~libssh2~nghttp2 build_system=autotools libs=shared,static tls=openssl arch=linux-sles15-zen2
[^] eyz7fdy ^pkg-config@0.29.2%intel@19.1.3.304+internal_glib build_system=autotools arch=linux-sles15-zen2
[^] zyh3dps ^git@2.35.3%intel@19.1.3.304+man+nls+perl+subtree~svn~tcltk build_system=autotools arch=linux-sles15-zen2
[^] 2mozkbq ^hdf5@1.14.0%intel@19.1.3.304~cxx+fortran+hl~ipo~java+mpi+shared+szip~threadsafe+tools api=default build_system=cmake build_type=RelWithDebInfo generator=make patches=0b5dd6f arch=linux-sles15-zen2
[^] othsdey ^libaec@1.0.6%intel@19.1.3.304~ipo+shared build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] qmbjrle ^nccmp@1.9.0.1%intel@19.1.3.304~ipo build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] kcnfazd ^netcdf-c@4.9.2%intel@19.1.3.304~byterange+dap~fismahigh~fsync~hdf4~jna+mpi+nczarr+optimize~parallel-netcdf+pic+shared+zstd build_system=autotools arch=linux-sles15-zen2
[^] d3jbtqt ^libxml2@2.10.3%intel@19.1.3.304~pic~python+shared build_system=autotools arch=linux-sles15-zen2
[^] q5chqll ^xz@5.4.1%intel@19.1.3.304~pic build_system=autotools libs=shared,static arch=linux-sles15-zen2
[^] ixnrcrh ^m4@1.4.19%intel@19.1.3.304+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-sles15-zen2
[^] ndc3de6 ^libsigsegv@2.14%intel@19.1.3.304 build_system=autotools arch=linux-sles15-zen2
[^] ssitvbp ^zstd@1.5.2%intel@19.1.3.304~ipo+programs build_system=cmake build_type=RelWithDebInfo compression=none generator=make libs=shared,static arch=linux-sles15-zen2
[^] p5o4oan ^netcdf-fortran@4.6.0%intel@19.1.3.304~doc+pic+shared build_system=autotools arch=linux-sles15-zen2
[^] nwb7voi ^parallel-netcdf@1.12.2%intel@19.1.3.304~burstbuffer+cxx+fortran+pic+shared build_system=autotools arch=linux-sles15-zen2
[^] 5pbpbxm ^perl@5.26.1%intel@19.1.3.304~cpanm+open+shared+threads build_system=generic patches=0eac10e,8cf4302 arch=linux-sles15-zen2
- d6ynrke ^parallelio@2.5.10%intel@19.1.3.304+fortran~ipo~logging+mpi+pnetcdf+shared~timing build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] owu3x4l ^py-pip@23.0%intel@19.1.3.304 build_system=generic arch=linux-sles15-zen2
[^] 3c7zodg ^python@3.8.6%intel@19.1.3.304+bz2+crypt+ctypes+dbm~debug+libxml2+lzma~nis~optimizations+pic+pyexpat+pythoncmd+readline+shared+sqlite3+ssl~tkinter+uuid+zlib build_system=generic patches=0d98e93,4c24573,f2fd060 arch=linux-sles15-zen2
[^] npeu4il ^wget@1.21.2%intel@19.1.3.304~libpsl~pcre~python+zlib build_system=autotools ssl=openssl arch=linux-sles15-zen2
[^] s5h77fy ^gettext@0.19.7%intel@19.1.3.304+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools patches=9acdb4e arch=linux-sles15-zen2
[^] 5le2wth ^bzip2@1.0.8%intel@19.1.3.304~debug~pic+shared build_system=generic arch=linux-sles15-zen2
[^] k4wj2dv ^tar@1.34%intel@19.1.3.304 build_system=autotools zip=pigz arch=linux-sles15-zen2
[^] xiq35gw ^pigz@2.7%intel@19.1.3.304 build_system=makefile arch=linux-sles15-zen2
[^] wxw7nrq ^libiconv@1.17%intel@19.1.3.304 build_system=autotools libs=shared,static arch=linux-sles15-zen2
[^] 74d3okx ^zlib@1.2.13%intel@19.1.3.304+optimize+pic+shared build_system=makefile arch=linux-sles15-zen2
[^] rbb3ovu ^crtm@2.4.0%intel@19.1.3.304+fix~ipo build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] 5rbf7lv ^crtm-fix@2.4.0_emc%intel@19.1.3.304+big_endian~little_endian+netcdf build_system=generic arch=linux-sles15-zen2
[^] nqhvyes ^git-lfs@2.11.0%intel@19.1.3.304 build_system=makefile arch=linux-sles15-zen2
- por67on ^esmf@8.4.2%intel@19.1.3.304~debug~external-lapack+external-parallelio+mpi+netcdf~pnetcdf~shared~xerces build_system=makefile esmf_comm=mpich3 esmf_os=Linux esmf_pio=auto patches=f63d405 snapshot=b09 arch=linux-sles15-zen2
[^] zhz3fn4 ^cray-mpich@8.1.9%intel@19.1.3.304~wrappers build_system=generic arch=linux-sles15-zen2
[^] uiitoco ^fms@2023.01%intel@19.1.3.304+gfs_phys~ipo+openmp+pic+quad_precision~yaml build_system=cmake build_type=Release constants=GFS generator=make precision=32,64 arch=linux-sles15-zen2
[^] hdaoete ^g2@3.4.5%intel@19.1.3.304~ipo+pic build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] 5znjjrh ^jasper@2.0.32%intel@19.1.3.304+jpeg~opengl+shared build_system=generic build_type=Release arch=linux-sles15-zen2
[^] bell45a ^libjpeg-turbo@2.1.0%intel@19.1.3.304~ipo~jpeg8+pic+shared+static build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] vtnwczf ^nasm@2.15.05%intel@19.1.3.304 build_system=autotools arch=linux-sles15-zen2
[^] s6zgkie ^libpng@1.6.37%intel@19.1.3.304~ipo~pic build_system=cmake build_type=RelWithDebInfo generator=make libs=shared,static arch=linux-sles15-zen2
[^] jiuagkl ^g2tmpl@1.10.2%intel@19.1.3.304~ipo build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] txgtete ^ip@3.3.3%intel@19.1.3.304~ipo+openmp+pic build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
- gs7uozx ^mapl@2.35.2%intel@19.1.3.304~debug+esma_gfe_namespace~extdata2g~fargparse~flap~ipo~pflogger+pnetcdf~shared build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] qfwcbi4 ^ecbuild@3.7.2%intel@19.1.3.304~fismahigh~ipo build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] 57l3e6y ^gftl-shared@1.5.0%intel@19.1.3.304~ipo build_system=cmake build_type=Release generator=make arch=linux-sles15-zen2
[^] yc64hit ^gftl@1.8.3%intel@19.1.3.304~ipo build_system=cmake build_type=Release generator=make arch=linux-sles15-zen2
[^] gdxl4uw ^yafyaml@0.5.1%intel@19.1.3.304~fismahigh~ipo build_system=cmake build_type=Release generator=make arch=linux-sles15-zen2
[^] lygv3cz ^scotch@7.0.3%intel@19.1.3.304+compression~esmumps~int64~ipo+metis+mpi~mpi_thread+noarch~shared~threads build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] j7z5z56 ^bison@3.8.2%intel@19.1.3.304 build_system=autotools arch=linux-sles15-zen2
[^] rvbxqfe ^diffutils@3.9%intel@19.1.3.304 build_system=autotools arch=linux-sles15-zen2
[^] eakmqwr ^flex@2.6.4%intel@19.1.3.304+lex~nls build_system=autotools patches=f8b85a0 arch=linux-sles15-zen2
[^] gyh7eic ^sp@2.3.3%intel@19.1.3.304~ipo+openmp+pic build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
[^] 5fd5vgl ^ufs-pyenv@1.0.0%intel@19.1.3.304 build_system=bundle arch=linux-sles15-zen2
[^] y5ruw2g ^py-cftime@1.0.3.4%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] lzo5wpv ^py-setuptools@59.4.0%intel@19.1.3.304 build_system=generic arch=linux-sles15-zen2
[^] w7rkpic ^py-wheel@0.37.1%intel@19.1.3.304 build_system=generic arch=linux-sles15-zen2
[^] jayhyji ^py-cython@0.29.33%intel@19.1.3.304 build_system=python_pip patches=71de066 arch=linux-sles15-zen2
[^] r3f3n2o ^py-f90nml@1.4.3%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] chdebgm ^py-jinja2@3.1.2%intel@19.1.3.304~i18n build_system=python_pip arch=linux-sles15-zen2
[^] pfbbicd ^py-markupsafe@2.1.1%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] 4h4hbqb ^py-netcdf4@1.5.3%intel@19.1.3.304~mpi build_system=python_pip patches=255b5ae arch=linux-sles15-zen2
[^] n3nete7 ^py-numpy@1.22.3%intel@19.1.3.304+blas+lapack build_system=python_pip patches=873745d,e9508c3 arch=linux-sles15-zen2
[^] xnvkgii ^openblas@0.3.19%intel@19.1.3.304~bignuma~consistent_fpcsr~ilp64+locking+noavx512+pic+shared build_system=makefile patches=114f95f symbol_suffix=none threads=none arch=linux-sles15-zen2
[^] ju2llg3 ^py-pandas@1.4.0%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] f6ri2gl ^py-bottleneck@1.3.5%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] kxmveh2 ^py-versioneer@0.28%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] fithdis ^py-numexpr@2.8.3%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] hyzrorv ^py-packaging@23.0%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] lwm3lmx ^py-flit-core@3.7.1%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] exqvnji ^py-pytz@2022.2.1%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] gzq5il6 ^py-python-dateutil@2.8.2%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] 4a47zrj ^py-setuptools-scm@7.0.5%intel@19.1.3.304+toml build_system=python_pip arch=linux-sles15-zen2
[^] 6xkt3o3 ^py-tomli@2.0.1%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] aebkj6y ^py-typing-extensions@4.5.0%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] kjrzpvl ^py-six@1.16.0%intel@19.1.3.304 build_system=python_pip arch=linux-sles15-zen2
[^] gteogol ^py-pyyaml@6.0%intel@19.1.3.304+libyaml build_system=python_pip arch=linux-sles15-zen2
[^] 25oabev ^libyaml@0.2.5%intel@19.1.3.304 build_system=autotools arch=linux-sles15-zen2
[^] jw353gp ^w3emc@2.9.2%intel@19.1.3.304~ipo+pic build_system=cmake build_type=RelWithDebInfo generator=make arch=linux-sles15-zen2
@mark-a-potts this is running or completed on all machines except azure and s4. do you have s4 access to do this?
on azure, it is not concretizing properly (for some reason it wants to modify basically all the library hashes), and i'm not sure why. since you set up the original 1.4.1 env there, do you mind taking a look (/contrib/EPIC/spack-stack/spack-stack-1.4.1/envs/ufs-pio-2.5.10
)?
@ulmononian I have found that if I can get an install done on one of the PW machines, I can usually take the spack.lock file from that machine and use it directly on the others after a "spack env activate ..." followed by "spack install". On several of them, you do have to run that "module unuse" command to get rid of the fake CRAY paths in the MODULEPATH.
@mark-a-potts ok i can give that a try (i.e., take an aws/gcp spack.lock and move to azure). thanks mark!
this is done on all major platforms.
@AlexanderRichert-NOAA do you want me to update the modulefiles in ufs-wm 1707 to use this ufs-pio-2.5.10 env?
Yep let's do it
nike'd
Is S4 done?
Is S4 done?
no. no one had access to the proper space to do the install.
Is S4 done?
no. no one had access to the proper space to do the install.
Ok. I can do that Monday if it's still needed.
@mark-a-potts @ulmononian @jkbk2004 I created the S4 chained environment with the latest modifications from @AlexanderRichert-NOAA per his email:
Change the spec entry in spack.yaml to:
ufs-weather-model-env%intel~python ^mapl build_type=Release ^esmf fflags="-fp-model precise" cxxflags="-fp-model precise" ^parallelio@2.5.10
Please use module use /data/prod/jedi/spack-stack/spack-stack-1.4.1/envs/ufs-pio-2.5.10/install/modulefiles/Core
, the rest is the same as for spack-stack-1.4.1.
Please let me know if this works for you. Thanks!
I haven't heard back in this issue, but I think the S4 install is correct. There is a separate ufs-weather-model issue that deals with testing and updating the ufs-w-m code to use those modules.
@climbfuji sorry -- i cannot use s4, so i can't check this! @DavidHuber-NOAA did make this comment on the ufs-wm pr: https://github.com/ufs-community/ufs-weather-model/pull/1707#issuecomment-1682225735, which reports that the RTs pass w/ what i think is this stack w/ two minor changes (loading git & nccmp).
@climbfuji @AlexanderRichert-NOAA this needs to be done for c5 and hercules since the ufs-wm cm group wants to move on https://github.com/ufs-community/ufs-weather-model/pull/1733 and https://github.com/ufs-community/ufs-weather-model/pull/1784. since i am not sure if these PRs will be added to the WM commit queue before or after rollout of spack-stack 1.5.0, we should probably append the c5/hercules installations to be safe. should i open an issue for this or just go ahead?
Yeah that makes sense. If you want to reopen this issue and add to the checklist then we'll have a record of it here.
I can help with one or both of the platforms, but Hercules is currently down for maintenance.
@AlexanderRichert-NOAA cool -- i'll re-open and add these two machines. @climbfuji i am already in the c5 stack space, so i will just do that one today. do you want to run hercules when it's back up? (it's actually the priority from the wm cm perspective i believe).
Yes I can do hercules, I am working on that platform these days anyway.
gaea c5 done.
Hercules is really stubborn. Something is wrong with the spack-stack-1.4.1 code/tree, because the concretizer refuses to work there. I am compiling develop as of today (which has pio 2.5.10 and all the other versions as the ufs-weather-model wants them). Let's just use this for the remainder of the time on Hercules until we switch to spack-stack-1.5.0?
sounds like the twelve labors did not end with retrieving cerberus after all. resisting concretization may need to be added as the 13th (@ hesiod).
at this point, this sounds like the best option. thanks for doing this!
p.s.
i will hold hercules RT testing for https://github.com/ufs-community/ufs-weather-model/pull/1733 until this new stack you are installing is ready (fyi @zach1221).
I do have a new environment with pio 2.5.10, but I realized that the ESMF and MAPL mods (basically it's only ESMF, but ...) were not in develop yet. So I need to create a chained env off of the just created unified-env. I'll name it ufs-pio-2.5.10 for consistency. It's worth remembering that this is ahead of spack-stack-1.4.1, therefore we shouldn't be surprised if we see b4b differences on other platforms when we move to 1.5.0 that don't show up on Hercules.
@AlexanderRichert-NOAA I will note that -fp-model
is only an intel option, therefore won't set this for gcc.
@ulmononian Hercules finally done:
The usual stuff for both Intel/GNU:
module purge
module use /work/noaa/epic/role-epic/spack-stack/hercules/modulefiles
module load ecflow/5.8.4
module load mysql/8.0.31
For Intel:
module use /work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-dev-20230825/envs/ufs-pio-2.5.10/install/modulefiles/Core
module load stack-intel/2021.9.0
module load stack-intel-oneapi-mpi/2021.9.0
module load stack-python/3.9.14
module load ufs-weather-model-env/1.0.0
For GNU:
module use /work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-dev-20230825/envs/ufs-pio-2.5.10/install/modulefiles/Core
module load stack-gcc/11.3.1
module load stack-openmpi/4.1.5
module load stack-python/3.9.14
module load ufs-weather-model-env/1.0.0
Please give this a try and let me know. Thanks!
@ulmononian Can we close this as completed?
@ulmononian Can we close this as completed?
yes!
Yay. Thanks!
Once we confirm that no other changes are needed, we need to do an interim installation of ufs-weather-model dependencies that uses parallelio@2.5.10 for https://github.com/ufs-community/ufs-weather-model/pull/1745 using spack-stack-1.4.1 as the basis (i.e., create another environment under the existing spack-stack-1.4.1 envs/ directory on each system).
Instructions:
Systems:
/glade/work/epicufsrt/contrib/spack-stack/cheyenne/spack-stack-1.4.1/envs/ufs-pio-2.5.10
)/lustre/f2/dev/role.epic/contrib/spack-stack/spack-stack-1.4.1-c4/envs/ufs-pio-2.5.10
)/scratch1/NCEPDEV/nems/role.epic/spack-stack/spack-stack-1.4.1/envs/ufs-pio-2.5.10
)/mnt/lfs4/HFIP/hfv3gfs/role.epic/spack-stack/spack-stack-1.4.1/envs/ufs-pio-2.5.10
)/contrib/EPIC/spack-stack/spack-stack-1.4.1/envs/ufs-pio-2.5.10
)/work/noaa/epic/role-epic/spack-stack/spack-stack-1.4.1/envs/ufs-pio-2.5.10
)/data/prod/jedi/spack-stack/spack-stack-1.4.1/envs/ufs-pio-2.5.10
)/work/noaa/epic/role-epic/spack-stack/hercules/spack-stack-dev-20230825
)/lustre/f2/dev/wpo/role.epic/contrib/spack-stack/c5/spack-stack-dev-20230717/envs/ufs-pio-2.5.10
)