search

ACCESS-NRI / ACCESS-OM2

ACCESS-OM2: ACCESS Ocean-Sea Ice Model
Apache License 2.0
5 stars 0 forks source link

Add developer docs #63

Closed aidanheerdegen closed 1 month ago

aidanheerdegen commented 2 months ago

Initial version to address #58

aidanheerdegen commented 2 months ago

Currently the dual component development build errors:

==> Error: ProcessError: Command exited with status 2:                                                                                                         
    '/g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/spack-build.sh' 'auscom' '360x300' '24x1' '24'                                        

4 errors found in build log:                                                                                                                                   
     186    cd /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/build_auscom_360x300_24x1_24p                                                   
     187    cat                                                                                                                                               
     188    cc -o makdep /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c                                                          
     189    In file included from /usr/include/bits/floatn.h(119),                                                                                             
     190                     from /usr/include/stdlib.h(55),                                                                                                  
     191                     from /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c(20):                                            
  >> 192    /usr/include/bits/floatn-common.h(214): error: invalid combination of type specifiers                                                             
     193      typedef float _Float32;                                                                                                                          
     194                    ^                                                                                                                                 
     195                                                                                                                                                      
     196    In file included from /usr/include/bits/floatn.h(119),                                                                          
     197                     from /usr/include/stdlib.h(55),                                                                                                  
     198                     from /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c(20):                                             
  >> 199    /usr/include/bits/floatn-common.h(251): error: invalid combination of type specifiers                                                              
     200      typedef double _Float64;                                                                                                                         
     201                     ^                                                                                                                                
     202                                                                                                                                                      
     203    In file included from /usr/include/bits/floatn.h(119),                                                                                            
     204                     from /usr/include/stdlib.h(55),                                                                                             
     205                     from /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c(20):                                             
  >> 206    /usr/include/bits/floatn-common.h(268): error: invalid combination of type specifiers                                                             
     207      typedef double _Float32x;                                                                                                                       
     208                     ^                                                                                                                    
     209                                                                                                                                                       
     210    In file included from /usr/include/bits/floatn.h(119),                                                                                            
     211                     from /usr/include/stdlib.h(55),                                                                                                  
     212                     from /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c(20):
  >> 213    /usr/include/bits/floatn-common.h(285): error: invalid combination of type specifiers                                                              
     214      typedef long double _Float64x;                                                                                                                  
     215                          ^                                                                                                                           
     216                                                                                                                                    
     217    compilation aborted for /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/bld/makdep.c (code 2)                                      
     218    exit 2                                                                                                                                             

See build log for details:                                                                                                                                     
  /g/data/.../dev_instructions/envs/mom5-cice5-dev/cice5/spack-build-out.txt

There are similar errors for the mom5 build.

aidanheerdegen commented 2 months ago

Another problem I came across: the instructions only seem to work for references like mom5@git.2023.11.09, when I tried mom5@git.master I got errors like

$ spack concretize -f                                                                                                                                                                     
==> Error: concretization failed for the following reasons:                                                                                                   

   1. cannot satisfy a requirement for package 'mom5'.
aidanheerdegen commented 2 months ago

Another problem I came across: the instructions only seem to work for references like mom5@git.2023.11.09, when I tried mom5@git.master I got errors

Now I think about it, maybe that is because I still had a package: depends in there from the deployment spack.yaml.

Which brings up another point. Should we provide a dev.yaml with less in it, and all the components already added as a spec to make it simpler for developers to create a suitable environment?

aidanheerdegen commented 2 months ago

Seems good to me so far. Some small changes.

Thanks for the review. What do you think structurally? Covers the right ground? Too much detail? Does there need to be a TL;DR at the top that just says how to get going super quickly?

Is it worth including all the spack output? It helps in some ways, but also creates a lot of visual clutter. Also the way it is done means they can't cut n' paste those code blocks and just run them for themselves.

Any opinion on providing a dev environment spack.yaml?

CodeGat commented 2 months ago

TLDRs are always good. Now that you mention it, the logging is excessive. Could possibly just do something like 

# output trimmed for brevity...

On some of the logging. For example, a lot of the [+] lines are redundant - just showing that packages have been installed earlier

aekiss commented 2 months ago

This is looking great, thanks guys. Could this document also state explicitly where to find the newly-compiled executable and how to add it to config.yaml so it can be run? (Does anything else need to change in config.yaml?)

harshula commented 2 months ago

The following worked:

[root@1cb2279fcb5a /]# git clone https://github.com/ACCESS-NRI/ACCESS-OM2.git
Cloning into 'ACCESS-OM2'...
remote: Enumerating objects: 250, done.
remote: Counting objects: 100% (60/60), done.
remote: Compressing objects: 100% (36/36), done.
remote: Total 250 (delta 43), reused 30 (delta 24), pack-reused 190
Receiving objects: 100% (250/250), 84.42 KiB | 6.03 MiB/s, done.
Resolving deltas: 100% (94/94), done.

[root@1cb2279fcb5a /]# mkdir mom5-dev

[root@1cb2279fcb5a /]# cd mom5-dev

[root@1cb2279fcb5a mom5-dev]# spack env create -d . ../ACCESS-OM2/spack.yaml
==> Created environment in /mom5-dev
==> You can activate this environment with:
==>   spack env activate /mom5-dev

[root@1cb2279fcb5a mom5-dev]# spack env activate .

[root@1cb2279fcb5a mom5-dev]# spack env status
==> Using spack.yaml in current directory: /mom5-dev

[root@1cb2279fcb5a mom5-dev]# spack develop mom5@git.2023.11.09

==> Configuring spec mom5@git.2023.11.09=2023.11.09 for development at path mom5

[root@1cb2279fcb5a mom5-dev]# spack concretize -f

==> Concretized access-om2@git.2024.03.0=2024.03.0
 -   75s5qbm  access-om2@git.2024.03.0=2024.03.0%intel@2021.2.0~deterministic build_system=bundle arch=linux-rocky8-x86_64
 -   2tvb2i6      ^cice5@git.2023.10.19=2023.10.19%intel@2021.2.0~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64
 -   ojigw3h          ^datetime-fortran@1.7.0%intel@2021.2.0 build_system=autotools patches=80b9577 arch=linux-rocky8-x86_64
[+]  l7h7kzk          ^gmake@4.4.1%intel@2021.2.0~guile build_system=generic arch=linux-rocky8-x86_64
 -   iov3ic2          ^netcdf-c@4.7.4%intel@2021.2.0~blosc~byterange~dap~fsync~hdf4~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared~szip~zstd build_system=autotools arch=linux-rocky8-x86_64
 -   2zxw3rg              ^hdf5@1.14.3%intel@2021.2.0~cxx~fortran+hl~ipo~java~map+mpi+shared~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
 -   w6fac4x          ^netcdf-fortran@4.5.2%intel@2021.2.0~doc+pic+shared build_system=autotools patches=b050dbd arch=linux-rocky8-x86_64
 -   v2ddwt6          ^oasis3-mct@git.2023.11.09=2023.11.09%intel@2021.2.0~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64
[+]  zsvealv          ^openmpi@4.0.2%intel@2021.2.0~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix+romio+rsh~singularity+static+vt+wrapper-rpath build_system=autotools fabrics=none patches=073477a,60ce20b schedulers=none arch=linux-rocky8-x86_64
[+]  m7o4usy              ^hwloc@2.9.1%intel@2021.2.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml~oneapi-level-zero~opencl+pci~rocm build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]  arw22dt                  ^libpciaccess@0.17%intel@2021.2.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  5x73eit                      ^util-macros@1.19.3%intel@2021.2.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  ztbumxg                  ^libxml2@2.10.3%intel@2021.2.0+pic~python+shared build_system=autotools arch=linux-rocky8-x86_64
[+]  6czv5xg                      ^libiconv@1.17%intel@2021.2.0 build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]  mamryan                      ^xz@5.4.1%intel@2021.2.0~pic build_system=autotools libs=shared,static arch=linux-rocky8-x86_64
[+]  hi4pfkb              ^numactl@2.0.14%intel@2021.2.0 build_system=autotools patches=4e1d78c,62fc8a8,ff37630 arch=linux-rocky8-x86_64
[+]  acsxl2d                  ^autoconf@2.69%intel@2021.2.0 build_system=autotools patches=35c4492,7793209,a49dd5b arch=linux-rocky8-x86_64
[+]  ulelybd                  ^automake@1.16.5%intel@2021.2.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  aibo2hi                  ^libtool@2.4.6%intel@2021.2.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  bl5vgn2                  ^m4@1.4.19%intel@2021.2.0+sigsegv build_system=autotools patches=9dc5fbd,bfdffa7 arch=linux-rocky8-x86_64
[+]  3qlauq6                      ^diffutils@3.9%intel@2021.2.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  nxptqrm                      ^libsigsegv@2.14%intel@2021.2.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  ijsfwt7              ^openssh@9.5p1%intel@2021.2.0+gssapi build_system=autotools arch=linux-rocky8-x86_64
[+]  zbzsece                  ^krb5@1.20.1%intel@2021.2.0+shared build_system=autotools arch=linux-rocky8-x86_64
[+]  oljratz                      ^bison@3.8.2%intel@2021.2.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  wxdsanc                      ^findutils@4.9.0%intel@2021.2.0 build_system=autotools patches=440b954 arch=linux-rocky8-x86_64
[+]  ulr3uy3                      ^gettext@0.22.3%intel@2021.2.0+bzip2+curses+git~libunistring+libxml2+pic+shared+tar+xz build_system=autotools arch=linux-rocky8-x86_64
[+]  m4kzokr                          ^bzip2@1.0.8%intel@2021.2.0~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64
[+]  7ohhm7g                          ^tar@1.34%intel@2021.2.0 build_system=autotools zip=pigz arch=linux-rocky8-x86_64
[+]  kw2iugs                              ^pigz@2.7%intel@2021.2.0 build_system=makefile arch=linux-rocky8-x86_64
[+]  clqsbru                              ^zstd@1.5.5%intel@2021.2.0+programs build_system=makefile compression=none libs=shared,static arch=linux-rocky8-x86_64
[+]  mgpwduu                  ^libedit@3.1-20210216%intel@2021.2.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  b3wuw76                  ^libxcrypt@4.4.35%intel@2021.2.0~obsolete_api build_system=autotools patches=4885da3 arch=linux-rocky8-x86_64
[+]  obhhhkg                  ^openssl@3.1.3%intel@2021.2.0~docs+shared build_system=generic certs=mozilla arch=linux-rocky8-x86_64
[+]  noqg2rf                      ^ca-certificates-mozilla@2023-05-30%intel@2021.2.0 build_system=generic arch=linux-rocky8-x86_64
[e]  u7fofm2              ^perl@5.26.3%intel@2021.2.0~cpanm+opcode+open+shared+threads build_system=generic patches=8cf4302 arch=linux-rocky8-x86_64
[+]  2tsjfk3              ^pmix@4.2.2%intel@2021.2.0~docs+pmi_backwards_compatibility~python~restful build_system=autotools arch=linux-rocky8-x86_64
[+]  r5biad2                  ^libevent@2.1.12%intel@2021.2.0+openssl build_system=autotools arch=linux-rocky8-x86_64
[+]  7swvjd2              ^zlib-ng@2.1.4%intel@2021.2.0+compat+opt build_system=autotools arch=linux-rocky8-x86_64
 -   taofugt          ^parallelio@2.5.2%intel@2021.2.0+fortran~ipo~logging+mpi~pnetcdf~shared~timing build_system=cmake build_type=Release generator=make patches=55a6d7a arch=linux-rocky8-x86_64
 -   xs5nghf      ^libaccessom2@git.2023.10.26=2023.10.26%intel@2021.2.0~deterministic~ipo~optimisation_report build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
[+]  qlvugmf          ^cmake@3.24.2%intel@2021.2.0~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rocky8-x86_64
[+]  3vvvrhz              ^curl@8.4.0%intel@2021.2.0~gssapi~ldap~libidn2~librtmp~libssh~libssh2+nghttp2 build_system=autotools libs=shared,static tls=openssl arch=linux-rocky8-x86_64
[+]  hrjpfs2                  ^nghttp2@1.57.0%intel@2021.2.0 build_system=autotools arch=linux-rocky8-x86_64
[+]  3b277qb              ^ncurses@6.4%intel@2021.2.0~symlinks+termlib abi=none build_system=autotools arch=linux-rocky8-x86_64
 -   a2vylfw          ^json-fortran@8.3.0%intel@2021.2.0~ipo build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64
[+]  2ibagqa          ^pkgconf@1.9.5%intel@2021.2.0 build_system=autotools arch=linux-rocky8-x86_64
 -   q6evhui      ^mom5@git.2023.11.09=2023.11.09%intel@2021.2.0~deterministic~optimisation_report+restart_repro build_system=makefile dev_path=/mom5-dev/mom5 type=ACCESS-OM arch=linux-rocky8-x86_64

==> Updating view at /mom5-dev/.spack-env/view

[root@1cb2279fcb5a mom5-dev]# spack install
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/gmake-4.4.1-l7h7kzk72xrid2uwjc4xaupuvxmcsw7h
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/bzip2-1.0.8-m4kzokrqyxvn7aeppdoypno364gg6bby
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/xz-5.4.1-mamryan4pg6psyxxcyr4djwcxuietwps
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/zlib-ng-2.1.4-7swvjd22aiggqhf2eu5osnxcvahyrv7w
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/pkgconf-1.9.5-2ibagqa3rb5spjcoopnoxqhbpndalrsd
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/zstd-1.5.5-clqsbrujtgzvooh5sutsqoomxcv6vqn5
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libxcrypt-4.4.35-b3wuw76yov75k6rxbnmpoiplcycyhzby
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/numactl-2.0.14-hi4pfkbi3qlkkarineyk2o7qfohmr44h
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libiconv-1.17-6czv5xgtld5kdoab2rfsd6kbppoznzfm
==> Installing datetime-fortran-1.7.0-ojigw3hps67jzwhr5by4ldeq2ddevypi [10/37]
==> No binary for datetime-fortran-1.7.0-ojigw3hps67jzwhr5by4ldeq2ddevypi found: installing from source
==> Fetching https://github.com/wavebitscientific/datetime-fortran/releases/download/v1.7.0/datetime-fortran-1.7.0.tar.gz
==> Applied patch /opt/spack-packages/packages/datetime-fortran/0001-Enable-deterministic-builds-using-D-flag-for-ar.patch
==> datetime-fortran: Executing phase: 'autoreconf'
==> datetime-fortran: Executing phase: 'configure'
==> datetime-fortran: Executing phase: 'build'
==> datetime-fortran: Executing phase: 'install'
==> datetime-fortran: Successfully installed datetime-fortran-1.7.0-ojigw3hps67jzwhr5by4ldeq2ddevypi
  Stage: 3.26s.  Autoreconf: 0.00s.  Configure: 1.33s.  Build: 1.45s.  Install: 0.04s.  Post-install: 0.03s.  Total: 6.14s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/datetime-fortran-1.7.0-ojigw3hps67jzwhr5by4ldeq2ddevypi
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/pigz-2.7-kw2iugsyeuywllqhgcw2gcn6hoird2bw
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/openssl-3.1.3-obhhhkgf27fa2qh5f4ui2e5np4k4f6cg
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libpciaccess-0.17-arw22dtj7arogotgmvj5udwclr6h5ifb
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/nghttp2-1.57.0-hrjpfs2lx7evq6bxnworz4v6xnynp2zo
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/ncurses-6.4-3b277qbzudjq6dxnnbjjktjqkdn7oc3w
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libxml2-2.10.3-ztbumxg43utno4dbwbz4b626s32wxsgp
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/tar-1.34-7ohhm7ga5agbvkj23f3egescr6tl7kyh
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libevent-2.1.12-r5biad2gihqnxjxylaaw5rdo4v7nlu4m
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/curl-8.4.0-3vvvrhzgjh3xbduumo6e2l3xw3o6tgfs
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libedit-3.1-20210216-mgpwduuernhhu2qyennfzqj3w2ce3oj5
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/hwloc-2.9.1-m7o4usyq2rajo72swf2srp3v7yjbkrr2
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/gettext-0.22.3-ulr3uy3sebp6n7y6ai6zuttswhwejbys
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/cmake-3.24.2-qlvugmfoczvvhzssbm5mob46z4jmzlx7
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/pmix-4.2.2-2tsjfk3t337rdc4xkv5mgqvz2vhrhnzb
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/krb5-1.20.1-zbzseceiavw4axvxgnx7wqzi5dpy6p7q
==> Installing json-fortran-8.3.0-a2vylfwg5e6nzwva4s4xoxjdxid34fv6 [26/37]
==> No binary for json-fortran-8.3.0-a2vylfwg5e6nzwva4s4xoxjdxid34fv6 found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/5f/5fe9ad709a726416cec986886503e0526419742e288c4e43f63c1c22026d1e8a.tar.gz
==> No patches needed for json-fortran
==> json-fortran: Executing phase: 'cmake'
==> json-fortran: Executing phase: 'build'
==> json-fortran: Executing phase: 'install'
==> json-fortran: Successfully installed json-fortran-8.3.0-a2vylfwg5e6nzwva4s4xoxjdxid34fv6
  Stage: 3.97s.  Cmake: 0.33s.  Build: 2.66s.  Install: 0.07s.  Post-install: 0.06s.  Total: 7.10s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/json-fortran-8.3.0-a2vylfwg5e6nzwva4s4xoxjdxid34fv6
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/openssh-9.5p1-ijsfwt7iygrmdra7ir7sszgiigwiee2q
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/openmpi-4.0.2-zsvealvafpe45lbzzg7yf42oiompduwo
==> Installing hdf5-1.14.3-2zxw3rgf3ls4emchlblhjkb66smyns4x [29/37]
==> No binary for hdf5-1.14.3-2zxw3rgf3ls4emchlblhjkb66smyns4x found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/09/09cdb287aa7a89148c1638dd20891fdbae08102cf433ef128fd345338aa237c7.tar.gz
==> Ran patch() for hdf5
==> hdf5: Executing phase: 'cmake'
==> hdf5: Executing phase: 'build'
==> hdf5: Executing phase: 'install'
==> hdf5: Successfully installed hdf5-1.14.3-2zxw3rgf3ls4emchlblhjkb66smyns4x
  Stage: 5.51s.  Cmake: 20.89s.  Build: 1m 6.71s.  Install: 0.79s.  Post-install: 0.21s.  Total: 1m 34.28s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/hdf5-1.14.3-2zxw3rgf3ls4emchlblhjkb66smyns4x
==> Installing netcdf-c-4.7.4-iov3ic2jttncqfyzl2p4ofrw5krqcc5c [30/37]
==> No binary for netcdf-c-4.7.4-iov3ic2jttncqfyzl2p4ofrw5krqcc5c found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/99/99930ad7b3c4c1a8e8831fb061cb02b2170fc8e5ccaeda733bd99c3b9d31666b.tar.gz
==> No patches needed for netcdf-c
==> netcdf-c: Executing phase: 'autoreconf'
==> netcdf-c: Executing phase: 'configure'
==> netcdf-c: Executing phase: 'build'
==> netcdf-c: Executing phase: 'install'
==> netcdf-c: Successfully installed netcdf-c-4.7.4-iov3ic2jttncqfyzl2p4ofrw5krqcc5c
  Stage: 6.33s.  Autoreconf: 0.00s.  Configure: 18.55s.  Build: 27.52s.  Install: 0.74s.  Post-install: 0.03s.  Total: 53.27s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/netcdf-c-4.7.4-iov3ic2jttncqfyzl2p4ofrw5krqcc5c
==> Installing netcdf-fortran-4.5.2-w6fac4x63uugrl4v2o2wmr4vpoxvw66q [31/37]
==> No binary for netcdf-fortran-4.5.2-w6fac4x63uugrl4v2o2wmr4vpoxvw66q found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/b9/b959937d7d9045184e9d2040a915d94a7f4d0185f4a9dceb8f08c94b0c3304aa.tar.gz
==> Applied patch /opt/spack/var/spack/repos/builtin/packages/netcdf-fortran/no_parallel_build.patch
==> netcdf-fortran: Executing phase: 'autoreconf'
==> netcdf-fortran: Executing phase: 'configure'
==> netcdf-fortran: Executing phase: 'build'
==> netcdf-fortran: Executing phase: 'install'
==> netcdf-fortran: Successfully installed netcdf-fortran-4.5.2-w6fac4x63uugrl4v2o2wmr4vpoxvw66q
  Stage: 3.57s.  Autoreconf: 0.00s.  Configure: 14.33s.  Build: 33.86s.  Install: 0.31s.  Post-install: 0.05s.  Total: 52.23s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/netcdf-fortran-4.5.2-w6fac4x63uugrl4v2o2wmr4vpoxvw66q
==> Waiting for oasis3-mct-git.2023.11.09=2023.11.09-v2ddwt6mumin6lsx5nkfj6b74nq==> Installing oasis3-mct-git.2023.11.09=2023.11.09-v2ddwt6mumin6lsx5nkfj6b74nqzfae6 [32/37]
==> No binary for oasis3-mct-git.2023.11.09=2023.11.09-v2ddwt6mumin6lsx5nkfj6b74nqzfae6 found: installing from source
==> No patches needed for oasis3-mct
==> oasis3-mct: Executing phase: 'edit'
==> oasis3-mct: Executing phase: 'build'
==> oasis3-mct: Executing phase: 'install'
==> oasis3-mct: Successfully installed oasis3-mct-git.2023.11.09=2023.11.09-v2ddwt6mumin6lsx5nkfj6b74nqzfae6
  Stage: 4.29s.  Edit: 0.00s.  Build: 1m 30.16s.  Install: 0.02s.  Post-install: 0.04s.  Total: 1m 34.60s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/oasis3-mct-git.2023.11.09_2023.11.09-v2ddwt6mumin6lsx5nkfj6b74nqzfae6
==> Installing parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o [33/37]
==> No binary for parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/93/935bc120ef3bf4fe09fb8bfdf788d05fb201a125d7346bf6b09e27ac3b5f345c.tar.gz
==> Moving resource stage
    source: /tmp/root/spack-stage/resource-genf90-taofugtgphqhfk2jrrr47bvb5yp2ns4o/spack-src/
    destination: /tmp/root/spack-stage/spack-stage-parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o/spack-src/genf90
==> Moving resource stage
    source: /tmp/root/spack-stage/resource-CMake_Fortran_utils-taofugtgphqhfk2jrrr47bvb5yp2ns4o/spack-src/
    destination: /tmp/root/spack-stage/spack-stage-parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o/spack-src/CMake_Fortran_utils
==> Applied patch /opt/spack/var/spack/repos/builtin/packages/parallelio/remove_redefinition_of_mpi_offset.patch
==> parallelio: Executing phase: 'cmake'
==> parallelio: Executing phase: 'build'
==> parallelio: Executing phase: 'install'
==> parallelio: Successfully installed parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o
  Stage: 10.89s.  Cmake: 5.37s.  Build: 5.86s.  Install: 0.13s.  Post-install: 0.03s.  Total: 22.39s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/parallelio-2.5.2-taofugtgphqhfk2jrrr47bvb5yp2ns4o
==> Waiting for libaccessom2-git.2023.10.26=2023.10.26-xs5nghfcy53dpfgomzutr34iy==> Installing libaccessom2-git.2023.10.26=2023.10.26-xs5nghfcy53dpfgomzutr34iym4vpjkd [34/37]
==> No binary for libaccessom2-git.2023.10.26=2023.10.26-xs5nghfcy53dpfgomzutr34iym4vpjkd found: installing from source
==> Ran patch() for libaccessom2
==> libaccessom2: Executing phase: 'cmake'
==> libaccessom2: Executing phase: 'build'
==> libaccessom2: Executing phase: 'install'
==> libaccessom2: Successfully installed libaccessom2-git.2023.10.26=2023.10.26-xs5nghfcy53dpfgomzutr34iym4vpjkd
  Stage: 5.10s.  Cmake: 2.16s.  Build: 4.63s.  Install: 0.14s.  Post-install: 0.04s.  Total: 12.16s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/libaccessom2-git.2023.10.26_2023.10.26-xs5nghfcy53dpfgomzutr34iym4vpjkd
==> Installing mom5-git.2023.11.09=2023.11.09-q6evhuii4knpdh3f3wb6mizyh22e2kxn [35/37]
==> No binary for mom5-git.2023.11.09=2023.11.09-q6evhuii4knpdh3f3wb6mizyh22e2kxn found: installing from source
==> No patches needed for mom5
==> mom5: Executing phase: 'edit'
==> mom5: Executing phase: 'build'
==> mom5: Executing phase: 'install'
==> mom5: Successfully installed mom5-git.2023.11.09=2023.11.09-q6evhuii4knpdh3f3wb6mizyh22e2kxn
  Stage: 0.00s.  Edit: 0.01s.  Build: 4m 16.12s.  Install: 0.04s.  Post-install: 0.08s.  Total: 4m 16.40s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/mom5-git.2023.11.09_2023.11.09-q6evhuii4knpdh3f3wb6mizyh22e2kxn
==> Installing cice5-git.2023.10.19=2023.10.19-2tvb2i6vqfugcwuoq6zkgtoqr33xrug3 [36/37]
==> No binary for cice5-git.2023.10.19=2023.10.19-2tvb2i6vqfugcwuoq6zkgtoqr33xrug3 found: installing from source
==> No patches needed for cice5
==> cice5: Executing phase: 'edit'
==> cice5: Executing phase: 'build'
==> cice5: Executing phase: 'install'
==> cice5: Successfully installed cice5-git.2023.10.19=2023.10.19-2tvb2i6vqfugcwuoq6zkgtoqr33xrug3
  Stage: 7.66s.  Edit: 0.01s.  Build: 11m 39.08s.  Install: 0.04s.  Post-install: 0.07s.  Total: 11m 46.98s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/cice5-git.2023.10.19_2023.10.19-2tvb2i6vqfugcwuoq6zkgtoqr33xrug3
==> Waiting for access-om2-git.2024.03.0=2024.03.0-75s5qbm3peejwou6u7e237swm7wek==> Installing access-om2-git.2024.03.0=2024.03.0-75s5qbm3peejwou6u7e237swm7wek4aq [37/37]
==> No binary for access-om2-git.2024.03.0=2024.03.0-75s5qbm3peejwou6u7e237swm7wek4aq found: installing from source
==> No patches needed for access-om2
==> access-om2: Executing phase: 'install'
==> access-om2: Successfully installed access-om2-git.2024.03.0=2024.03.0-75s5qbm3peejwou6u7e237swm7wek4aq
  Stage: 0.00s.  Install: 0.00s.  Post-install: 0.05s.  Total: 0.14s
[+] /opt/release/linux-rocky8-x86_64/intel-2021.2.0/access-om2-git.2024.03.0_2024.03.0-75s5qbm3peejwou6u7e237swm7wek4aq
==> Updating view at /mom5-dev/.spack-env/view
harshula commented 2 months ago

I was able to successfully change the version of a dependency. e.g. Change the version of parallelio from 2.5.2 to 2.5.10 by doing:

1) Edit the environment's spack.yaml file. 2) Remove the section:

    parallelio:
      require: '@2.5.2'

3) Modify the view and concretizer to be (It should also work if you set view: false.):

  view:
    default:
      root: spack-view
      projections:
        all: '{name}/{version}'
  concretizer:
    unify: when_possible

4) Run:

spack add parallelio@2.5.10
spack concretize -f
spack install
aidanheerdegen commented 2 months ago

Thanks for the thorough review and feedback

The following worked:

[root@1cb2279fcb5a /]# git clone https://github.com/ACCESS-NRI/ACCESS-OM2.git
[root@1cb2279fcb5a mom5-dev]# spack env create -d . ../ACCESS-OM2/spack.yaml
[root@1cb2279fcb5a mom5-dev]# spack env activate .
[root@1cb2279fcb5a mom5-dev]# spack env status
[root@1cb2279fcb5a mom5-dev]# spack develop mom5@git.2023.11.09
[root@1cb2279fcb5a mom5-dev]# spack concretize -f
[root@1cb2279fcb5a mom5-dev]# spack install

I got an error at concretisation:

$ spack concretize -f
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.5/build_cache/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstr
ap-spack-bhqgwuvef354fwuxq7heeighavunpber.spec.json
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.5/build_cache/linux-centos7-x86_64/gcc-10.2.1/clingo-bootstr
ap-spack/linux-centos7-x86_64-gcc-10.2.1-clingo-bootstrap-spack-bhqgwuvef354fwuxq7heeighavunpber.spack
==> Installing "clingo-bootstrap@=spack%gcc@=10.2.1~docs+ipo+optimized+python+static_libstdcpp build_system=cmake build_type=
Release generator=make patches=bebb819,ec99431 arch=linux-centos7-x86_64" from a buildcache
==> Error: concretization failed for the following reasons:

   1. No valid value for variant 'deterministic' of package 'access-om2'

I am chaining the 0.20 spack tree to this install (though didn't document in instructions) like so

$ cat spack/etc/spack/upstreams.yaml 
upstreams:
  spack-vk83-0.20:
    install_tree: /g/data/vk83/apps/spack/0.20/release

If I delete that upstream it concretises ok. I guess that is due to a hash mismatches between 0.20 and 0.21? It worked when I was using spack v0.20.

Installation works, but is slow because it has to compile all the dependencies.

So I think I've established that we can't mix spack versions when chaining.

aidanheerdegen commented 2 months ago

I was able to successfully change the version of a dependency. e.g. Change the version of parallelio from 2.5.2 to 2.5.10

Thanks again for working through this. I can confirm that this installed parallel-2.5.10, but doesn't rebuild cice5 to use the updated dependency:

$ spack find --paths --deps cice5
==> In environment /g/data/tm70/aph502/dev_instructions/envs/mom5-dev
==> Root specs
access-om2@git.2024.03.0=2024.03.0

parallelio@2.5.10

==> Installed packages
-- linux-rocky8-x86_64_v4 / intel@19.0.5.281 --------------------
cice5@git.2023.10.19=2023.10.19             /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/cice5-git.2023.10.19_2023.10.19-ckoimsofpfnlwizd3q6zcdd6zsdbu2hv
    datetime-fortran@1.7.0                  /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/datetime-fortran-1.7.0-oy47sjopcbyjxejfinwe3ept564idest
    gmake@4.4.1                             /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/gmake-4.4.1-ggoluv3v37ruoyyjo4idshs6lcfcnpyb
    libaccessom2@git.2023.10.26=2023.10.26  /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/libaccessom2-git.2023.10.26_2023.10.26-hve4wjl273pya4j4s6dbvdjistrid2oa
        cmake@3.24.2                        /apps/cmake/3.24.2
        json-fortran@8.3.0                  /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/json-fortran-8.3.0-dlmgtabad5koosoqhjfcs7323itzzrxx
        pkgconf@1.9.5                       /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/pkgconf-1.9.5-3f2l3qwlgatzrt6kdcigbdiadyydy6ld
    netcdf-c@4.7.4                          /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/netcdf-c-4.7.4-lzrdcziljsdeosfzwpti2lkrad5ixdju
        hdf5@1.14.3                         /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/hdf5-1.14.3-45zs6fv3fmmilvlb753kfscuxlqmlrq4
            zlib-ng@2.1.4                   /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/zlib-ng-2.1.4-jugwcxhg3jwqkp4rzchowtra6zlmoxn3
    netcdf-fortran@4.5.2                    /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/netcdf-fortran-4.5.2-g3tf2uevw73rzufozccjav5bumvebkto
    oasis3-mct@git.2023.11.09=2023.11.09    /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/oasis3-mct-git.2023.11.09_2023.11.09-bhk6guqf66zqonmikcecpkpb3hmg4uaa
    openmpi@4.0.2                           /apps/openmpi/4.0.2
    parallelio@2.5.2                        /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/parallelio-2.5.2-ylrx5soeqhfqvdszal3dchfarzzk5wvq

==> 1 installed package

Tried

$ spack install --overwrite
==> The following package specs will be reinstalled:

-- linux-rocky8-x86_64_v4 / intel@19.0.5.281 --------------------
coujqai access-om2@git.2024.03.0=2024.03.0%intel ~deterministic build_system=bundle
==> Do you want to proceed? [y/N] y
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/datetime-fortran-1.7.0-oy47sjopcbyjxejfinwe3ept564idest
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/json-fortran-8.3.0-dlmgtabad5koosoqhjfcs7323itzzrxx
==> openmpi@4.0.2 : has external module in ['openmpi/4.0.2']
[+] /apps/openmpi/4.0.2 (external openmpi-4.0.2-skdxaycsjbzb33gqqp252d7vu6zuai5e)
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/pkgconf-1.9.5-3f2l3qwlgatzrt6kdcigbdiadyydy6ld
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/zlib-ng-2.1.4-jugwcxhg3jwqkp4rzchowtra6zlmoxn3
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/hdf5-1.14.3-45zs6fv3fmmilvlb753kfscuxlqmlrq4
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/netcdf-c-4.7.4-lzrdcziljsdeosfzwpti2lkrad5ixdju
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/netcdf-fortran-4.5.2-g3tf2uevw73rzufozccjav5bumvebkto
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/parallelio-2.5.2-ylrx5soeqhfqvdszal3dchfarzzk5wvq
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/oasis3-mct-git.2023.11.09_2023.11.09-bhk6guqf66zqonmikcecpkpb3hmg4uaa
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/libaccessom2-git.2023.10.26_2023.10.26-hve4wjl273pya4j4s6dbvdjistrid2oa
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/mom5-git.2023.11.09_2023.11.09-6m2cz6ai7znxxup7hvt5cornxgg7cwei
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/cice5-git.2023.10.19_2023.10.19-ckoimsofpfnlwizd3q6zcdd6zsdbu2hv
==> Installing access-om2-git.2024.03.0=2024.03.0-coujqair3vkzv5jickbilp5trnvjugi3 [14/14]
==> No binary for access-om2-git.2024.03.0=2024.03.0-coujqair3vkzv5jickbilp5trnvjugi3 found: installing from source
==> No patches needed for access-om2    
==> access-om2: Executing phase: 'install'
==> access-om2: Successfully installed access-om2-git.2024.03.0=2024.03.0-coujqair3vkzv5jickbilp5trnvjugi3
  Stage: 0.00s.  Install: 0.00s.  Post-install: 0.32s.  Total: 7.13s
[+] /g/data/tm70/aph502/dev_instructions/release/linux-rocky8-x86_64_v4/intel-19.0.5.281/access-om2-git.2024.03.0_2024.03.0-coujqair3vkzv5jickbilp5trnvjugi3
==> Warning: Module file /g/data/tm70/aph502/dev_instructions/release/modules/linux-rocky8-x86_64_v4/access-om2/2024.03.0 exists and will not be overwritten

But it didn't install a new version of cice5 with the new parallelio version.

Adding

    parallelio:
      require: '@2.5.10'

to the spack.yaml didn't change the outcome.

aidanheerdegen commented 1 month ago

It was becoming unwieldy trying to investigate the issues with multi-component and updating dependencies, so I've turned this into a draft so it isn't merged, but we retain all the review comments in place.

I've created a new PR: #67 to get a simpler version merged and work on the other examples individually.

harshula commented 1 month ago

Hi @aidanheerdegen , There's no target being set in https://github.com/ACCESS-NRI/ACCESS-OM2/blob/main/spack.yaml . You need PR https://github.com/ACCESS-NRI/ACCESS-OM2/pull/60 to be merged or use branch om2-dependencies-update to get target=x86_64.

harshula commented 1 month ago

Thanks again for working through this. I can confirm that this installed parallel-2.5.10, but doesn't rebuild cice5 to use the updated dependency:

First try doing spack uninstall --dependents parallelio@2.5.2 %<compiler>

harshula commented 1 month ago

Hi @aidanheerdegen , I was able to successfully change the version of a dependency and rebuild the dependents. e.g. Change the version of parallelio from 2.5.2 to 2.5.10:

1) Edit the environment's spack.yaml file and make the following change:

diff --git a/spack.yaml b/spack.yaml
index 0754065..c8f2ab0 100644
--- a/spack.yaml
+++ b/spack.yaml
@@ -20,7 +20,7 @@ spack:
     netcdf-fortran:
       require: '@4.5.2'
     parallelio:
-      require: '@2.5.2'
+      require: '@2.5.10'
     openmpi:
       require: '@4.0.2'
     all:

2) Run:

spack uninstall --dependents parallelio@2.5.2 %<compiler>
spack concretize -f
spack install