search

COSIMA / access-om2

Deprecated ACCESS-OM2 global ocean - sea ice coupled model code and configurations.
21 stars 23 forks source link

Use 1st order conservative weights #71

Closed nichannah closed 4 years ago

nichannah commented 6 years ago

We have a development snapshot of ESMF_RegridWeightGen with support for 2nd order conservative interpolation. We have been able to use this on the 1 and 0.25 deg configurations however at 0.1 it will run out of memory given any reasonable quantity of resources.

I've tried on megamem with 3Tb memory per node and several 1000 PEs.

For the time being we are using the 1st order conservative and patch schemes for this resolution but this problem should be revisited when the ESMF_RegridWeightGen 2nd order code has been officially released.

nichannah commented 5 years ago

The new version of ESMF_RegridWeightGen seems to do the job. I will copy over a new 0.1 weights files soon.

aekiss commented 5 years ago

2nd order conservative interpolation can go negative due to spurious overshoots so is not suitable for fields that are sign-definite. See https://github.com/OceansAus/access-om2/issues/74#issuecomment-454660871 https://github.com/OceansAus/access-om2/issues/74#issuecomment-459275271

Until this is resolved I think it would be best to use 1st order conservative interpolation for all fluxes and all resolutions (i.e. we should stop using 2nd order at 1 degree and 0.25 degree).

We still need smooth winds though: https://github.com/OceansAus/access-om2/issues/52

aekiss commented 5 years ago

@nichannah do we have any weights files at 1 deg and 0.25 deg that use 1st order conservative interpolation for all fluxes? Or will somebody need to create them following https://github.com/COSIMA/access-om2/wiki/Technical-documentation#creating-remapping-weights ?

aekiss commented 5 years ago

Hi @nichannah - I've had a look into this following your suggestion at TWG today.

The 2nd order conservative 0.25deg weight file currently being used is /short/public/access-om2/input_08022019/common_025deg_jra55/rmp_jra55_cice_conserve.nc which dates back to 22 Nov 2017.

The newest input before that date is input_c22b3335 which contains several 0.25deg rmp_*_CONSERV.nc possibilities:

input/oasis_jra55_to_025deg/rmp_jrar_to_cict_CONSERV.nc
input/oasis_jra55_to_025deg/rmp_cict_to_jrat_CONSERV.nc
input/oasis_jra55_to_025deg/rmp_jrat_to_cict_CONSERV.nc

they all use

                :map_method = "Conservative remapping" ;
                :ESMF_regrid_method = "First-order Conservative" ;

so I guess that's what we're looking for. I'll have a look in old namcouples to get an idea what these 3 files were used for.

aekiss commented 5 years ago

make_remap_weights.py apparently only supports patch and conserve2nd. So I think we should also support conserve (1st-order conservative) - e.g. https://github.com/COSIMA/access-om2/compare/esmf-conserve?expand=1

aekiss commented 5 years ago

Summary of 2nd order conservative remapping files that need to be changed to 1st order:

1deg_jra55_iaf/namcouple and 1deg_jra55_ryf/namcouple use common_1deg_jra55/rmp_jra55_cice_conserve.nc which is 2nd order conservative (despite the misleading name)

1deg_core_nyf/namcouple uses common_1deg_core/rmp_nt62_to_cice_CONSERV_FRACNNEI.nc which is conservative but at unspecified order. However it was created in 2013 so I think it's safe to assume that's 1st order. It also uses this for all fields other than wind, but something smooth would be better for air pressure, temperature and humidity. Wind uses common_1deg_core/rmp_nt62_to_cice_DISTWGT.nc which uses distance weighted avg of nearest neighbors, but other 1 deg configs use patch. I've broken this out as a separate issue: https://github.com/COSIMA/access-om2/issues/164

025deg_jra55_iaf/namcouple and 025deg_jra55_ryf/namcouple use common_025deg_jra55/rmp_jra55_cice_conserve.nc which is 2nd order conservative (despite the misleading name)

025deg_core2_nyf/namcouple uses common_025deg_core/rmp_core2_cice_conserve2nd.nc which is 2nd order conservative

All 0.1 deg configs use 1st order conservative.

aekiss commented 5 years ago

I've just created the first-order conservative /short/public/access-om2/input_rc/common_1deg_jra55/JRA55_MOM1_conserve.nc using this version of make_remap_weights.sh.

I don't know if that would be a suitable replacement for the 2nd order /short/public/access-om2/input_rc/common_1deg_jra55/rmp_jra55_cice_conserve.nc. There are some differences, e.g. in num_links, but perhaps that is to be expected.

@russfiedler, @nichannah - grateful for any insights here

aekiss commented 5 years ago

caveat: running make_remap_weights.py produced a lot of messages like

[raijin1:24039] shmem: posix: file name search - max attempts exceeded.cannot continue with posix.
aekiss commented 5 years ago

Also I've had no luck so far in generating 0.25 deg weights with /short/v45/aek156/sources/new/weights/access-om2/tools/make_remap_weights.sh, so I've tried to find an old 0.25 deg 1st order conservative weights file.

I had to go back to a namcouple from when control dirs weren't submodules, which used rmp_jrat_to_cict_CONSERV.nc but from oasis_jra55_to_025deg which has no hash so it's unclear what version it is. But presumably it's the same as rmp_jrat_to_cict_CONSERV.nc in /short/public/access-om2/input_c22b3335.tar, which uses First-order Conservative.

So I've copied that to /short/public/access-om2/input_rc/common_025deg_jra55/rmp_jrat_to_cict_CONSERV.nc.

nichannah commented 5 years ago

Need to review where we are at with this.

nichannah commented 5 years ago

There shouldn't be any problem with using the old remapping weights other than lack of consistency, documentation etc. I'll fix these.

aekiss commented 4 years ago

@nichannah this issue (specifically, using 1st order conservative weights for fluxes at 1 and 0.25 deg) is the only thing remaining to be done for /g/data4/ik11/inputs/access-om2/input_rc, which is intended as the new default set of inputs for all but the (now deprecated) core configs. /g/data4/ik11/inputs/access-om2/input_rc/README.txt shows the state of play.

Would you be able to take a look at this pls?

aekiss commented 4 years ago

closing - 1st order conservative weights are used in all the ak-dev JRA configs

access-hive-bot commented 1 year ago

This issue has been mentioned on ACCESS Hive Community Forum. There might be relevant details there:

https://forum.access-hive.org.au/t/initializing-a-1-4-degree-simulation-given-a-1-degree-temperature-salinity-data/403/2

aekiss commented 1 year ago

ESMF are planning to introduce monotonic 2nd order conservative regridding - not sure about the details or timeline.

access-hive-bot commented 1 month ago

This issue has been mentioned on ACCESS Hive Community Forum. There might be relevant details there:

https://forum.access-hive.org.au/t/extending-access-om2-025-ryf-runs-on-ik11/3741/12

access-hive-bot commented 1 month ago

This issue has been mentioned on ACCESS Hive Community Forum. There might be relevant details there:

https://forum.access-hive.org.au/t/extending-access-om2-025-ryf-runs-on-ik11/3741/14

access-hive-bot commented 1 month ago

This issue has been mentioned on ACCESS Hive Community Forum. There might be relevant details there:

https://forum.access-hive.org.au/t/extending-access-om2-025-ryf-runs-on-ik11/3741/15