search

NCAR / CESM_postprocessing

Project repository for the CESM python based post-processing code, documentation and issues tracking.
77 stars 45 forks source link

Lnd timeseries files created for hourly data have year overlaps #138

Closed olyson closed 6 years ago

olyson commented 6 years ago

I'm using the postprocessing to generate lnd timeseries files for hourly data. I'm requesting year chunks, i.e., my settings in env_timeseries.xml are:

  <file_extension suffix=".h7.[0-9]">
    <subdir>hist</subdir>
    <tseries_create>TRUE</tseries_create>
    <tseries_output_format>netcdf4c</tseries_output_format>
    <tseries_tper>hour_1</tseries_tper>
    <tseries_filecat_tper>years</tseries_filecat_tper>
    <tseries_filecat_n>1</tseries_filecat_n>
  </file_extension>

But I'm getting the following timeseries files, e.g., for the variable BTRAN:

clm50sp_clm50dev001_1deg_GSWP3V1_hist.clm2.h7.BTRAN.2005010100-2005123100.nc clm50sp_clm50dev001_1deg_GSWP3V1_hist.clm2.h7.BTRAN.2005010100-2006123100.nc clm50sp_clm50dev001_1deg_GSWP3V1_hist.clm2.h7.BTRAN.2006010100-2007123100.nc clm50sp_clm50dev001_1deg_GSWP3V1_hist.clm2.h7.BTRAN.2007010100-2008123100.nc clm50sp_clm50dev001_1deg_GSWP3V1_hist.clm2.h7.BTRAN.2008010100-2009123100.nc clm50sp_clm50dev001_1deg_GSWP3V1_hist.clm2.h7.BTRAN.2009010100-2010123100.nc clm50sp_clm50dev001_1deg_GSWP3V1_hist.clm2.h7.BTRAN.2010010100-2011123100.nc clm50sp_clm50dev001_1deg_GSWP3V1_hist.clm2.h7.BTRAN.2011010100-2012123100.nc clm50sp_clm50dev001_1deg_GSWP3V1_hist.clm2.h7.BTRAN.2012010100-2013123100.nc clm50sp_clm50dev001_1deg_GSWP3V1_hist.clm2.h7.BTRAN.2013010100-2014123100.nc

So, the first year (2005) is ok, but then I get a file with two years on it, and then the next file in the sequence has two years on it with a year of overlap with the previous file.

Am I missing a setting where or is something overriding my settings in env_timeseries.xml?

My postprocessing case directory is:

/glade/p/work/oleson/diagnostics/runs/clm50sp_clm50dev001_1deg_GSWP3V1_hist

bertinia commented 6 years ago

@olyson - I'm able to repeat the problem using your clm2.h7 data files for this case.

bertinia commented 6 years ago

For @sherimickelson: Here's my postprocess case:

/glade/p/work/aliceb/sandboxes/runs/clm50sp_clm50dev001_1deg_GSWP3V1_hist

and the raw input history files are here:

/glade2/scratch2/aliceb/clm50sp_clm50dev001_1deg_GSWP3V1_hist/lnd/hist

There is no difference between the ts_status.log. and timeseries.log. files for 1-year chunks or a single 10-year chunk. I'm running the 1-year chunks again with additional debugging information.

bertinia commented 6 years ago

This problem only appears for this dataset clm2.h7 1-hour data chunked into 1 year single variable time-series files. If the chunk size is set to 10 years, then there is no problem. The problem also seems to be unrelated to the PE decomposition as I tried with the default 4 nodes, 18 mpi-tasks-per-node and 1 node, 16 mpi-tasks-per-node on cheyenne.

bertinia commented 6 years ago

fixed with PR #139