Problems with updating the land fraction in historical run

[MartinDix]

The historical experiment calls a script to update the land fraction at the start of each run. This is dependent on the conda/analysis3 module. However when using the vk83 payu the gdata/hh5 storage directive isn't included and the script doesn't run correctly. The run itself doesn't fail at this point.

The script runs at the start of each run and modifies a copy of the restart file. This updates the field correctly for this run, but the restart in the archive is not consistent with what was actually run. The original script setup ran the land fraction update at the end of the year so that the restart for the next year was consistent. Ideally do the same here.

The script also needs updating to avoid some numpy deprecations.

[MartinDix]

Script would be better called as an archive script, though this would have been trickier before the appropriate environment variables became available, https://github.com/payu-org/payu/pull/445

[MartinDix]

Original CSIRO runs used LUC data from individual year files cableCMIP6_LC_1850.nc,cableCMIP6_LC_1851.nc etc. from /g/data/p66/txz599/data/luc_hist_maps. The time coordinate in all these files is the start of the next year for some reason

ncdump -t -v time  /g/data/p66/txz599/data/luc_hist_maps/cableCMIP6_LC_1850.nc 
 time = "1851-01-01" ;

The original CSIRO scripts replaced the LUC file in the restart at the end of each year, using the file for the next year. This didn't use the time in the file so the inconsistency didn't matter.

The payu setup uses a netCDF file with the years concatenated and uses the file times to select the field. /g/data/access/payu/access-esm/input/historical/atmosphere/cableCMIP6_LC_1850-2015.v1.nc is a simple concatenation so has the wrong times. This is corrected in cableCMIP6_LC_1850-2015.v2.nc

https://github.com/coecms/access-esm/commit/aa35ea7898559dec9128d3a8771b6b6f3b40792a switched to using data from /g/data/access/payu/access-esm/input/historical.corrected. The original directory had ancillary files with a modified initial date to address https://github.com/coecms/access-esm/issues/15. This would have given reproducibility for the first year but not for later years. The historical.corrected directory removes these changes (I think appropriately) and uses files identical to those from /g/data/p66/txz599/data/ancil/CMIP6. However it also includes the uncorrected cableCMIP6_LC_1850-2015.nc, so the land use changes lag by a year.

Correct approach is to use all files from the historical.corrected directory except for LUC and use cableCMIP6_LC_1850-2015.v2.nc for that.

[MartinDix]

Original script version used

python /g/data/p66/txz599/ACCESSESM1/scripts/um_replace_field_multilevel.py -v 835 -V fraction -n /g/data1a/p66/txz599/data/luc_hist_maps/$ncfile_prev ${restartfile} 
python /g/data/p66/txz599/ACCESSESM1/scripts/um_replace_field_multilevel.py -v 216 -V fraction -n /g/data1a/p66/txz599/data/luc_hist_maps/$ncfile ${restartfile}

so set field 835 to the previous year fraction and 216 to the new fraction. Current payu version is using scripts/update_cable_vegfrac.py that only sets field 216. There is also scripts/update_landuse.py that does the right thing. This was used originally but switched away from by https://github.com/coecms/access-esm/commit/1c34a6eef6bbe57dd80b0bb59bc26aa232b4bd02.

[MartinDix]

The cableCMIP6_LC_1850-2015.v2.nc file starts with 1851. update_landuse.py uses nearest matching on times and sets the previous fraction to the 1851 value because 1850 isn't available. The difference between new and previous fractions is used to adjust the carbon budgets so this would have affected results. Modified the script to use exact time matching so this error isn't possible.

Created a new cableCMIP6_LC_1850-2015.nc file that included 1850. Combining the original files with cdo copy doesn't work properly because they have fraction:valid_max = 1.f ; and some values slightly larger than 1 get converted to NaN. To maintain reproducibility removed this attribute rather than truncating the fractions at 1 (perhaps the model does this?).

Doing the concatenation and time adjustment in a single step with cdo copy -shifttime,-1years cableCMIP6_LC_????.nc cableCMIP6_LC_1850-2015.nc gives a messed up time axis for some reason

ncdump -t -v time cableCMIP6_LC_1850-2015.nc
time = "1850-01-01", "1852-01-01", ...

so instead split it with

cdo copy cableCMIP6_LC_????.nc temp.nc
cdo -shifttime,-1years temp.nc cableCMIP6_LC_1850-2015.nc

[MartinDix]

This compares the change in land fraction in the published CMIP6 data /g/data/fs38/publications/CMIP6/CMIP /CSIRO/ACCESS-ESM1-5/historical/r1i1p1f1/Lmon/landCoverFrac/gn/latest/landCoverFrac_Lmon_ACCESS-ESM1 -5_historical_r1i1p1f1_gn_185001-201412.nc from that in the new config 61f091b379ac664250aa72b5d03ddf0496263b8a