payu-org / payu

A workflow management tool for numerical models on the NCI computing systems
Apache License 2.0
18 stars 25 forks source link

Differing behavoir with `payu run` vs. `payu setup` #327

Closed rmholmes closed 2 years ago

rmholmes commented 2 years ago

I usually use payu setup to check my input files are linked correctly. However, in testing for the new ERA-5 runs, I've found that this check can be inconsistent with what payu run gives you.

E.g. in my run at /home/561/rmh561/access-om2/1deg_era5_iaf/ (or https://github.com/rmholmes/1deg_era5_iaf/commit/3436cd1ca19cb6efc64913cd9514f4c594caff69), I have in my config.yaml:

...
    - name: atmosphere
      model: yatm
      exe: /g/data/ik11/inputs/access-om2/bin/yatm_61b1f38.exe
      input:
            - /g/data/e14/rmh561/access-om2/input/ERA-5/
            - /g/data/ik11/inputs/access-om2/input_20210915/common_1deg_era5
 ...

When I do payu sweep && payu run I get:

[rmh561@gadi-login-01 1deg_era5_iaf]$ ls -lh work/atmosphere/INPUT/ERA5_MOM1_*
lrwxrwxrwx 1 rmh561 e14 84 Apr 13 16:59 work/atmosphere/INPUT/ERA5_MOM1_conserve.nc -> /g/data/ik11/inputs/access-om2/input_20210915/common_1deg_era5/ERA5_MOM1_conserve.nc
lrwxrwxrwx 1 rmh561 e14 81 Apr 13 16:59 work/atmosphere/INPUT/ERA5_MOM1_patch.nc -> /g/data/ik11/inputs/access-om2/input_20210915/common_1deg_era5/ERA5_MOM1_patch.nc

But when I just do payu setup (which is what I normally do to check things);

[rmh561@gadi-login-01 1deg_era5_iaf]$ ls -lh work/atmosphere/INPUT/ERA5_MOM1_*
lrwxrwxrwx 1 rmh561 e14 63 Apr 13 17:04 work/atmosphere/INPUT/ERA5_MOM1_conserve.nc -> /g/data/e14/rmh561/access-om2/input/ERA-5/ERA5_MOM1_conserve.nc
lrwxrwxrwx 1 rmh561 e14 62 Apr 13 17:04 work/atmosphere/INPUT/ERA5_MOM1_patch.nc -> /./g/data/e14/rmh561/access-om2/input/ERA-5/ERA5_MOM1_patch.nc

They're pointing at different files!

I suspect storage flags or permissions issues. Or I'm doing something stupid.

rmholmes commented 2 years ago

Never mind. This was my mistake. My files were linked to a location that wasn't included in the storage flags. Too many symbolic links!!