using multiple single gfs and hycom files error: nc file does not have a aggregation dimension

[limaolin1981611]

I used multiple single gfs and hycom files to run the leeway model. I have added the "standard_name" attributes to the files. The names of gfs and hycom files are as follows: gfs_1p00_sfc_wind_uv_Global_2020070600_t000.nc, gfs_1p00_sfc_wind_uv_Global_2020070600_t003.nc, ..., gfs_1p00_sfc_wind_uv_Global_2020070600_t072.nc and hycom_glb_sfc_water_uv_Global_2020070600_t000.nc, hycom_glb_sfc_water_uv_Global_2020070600_t003.nc, hycom_glb_sfc_water_uv_Global_2020070600_t072.nc. I used the wildcards simply like gfs_1p00_sfc_wind_uv_Global_2020070600_t0.nc and hycom_glb_sfc_water_uv_Global_2020070600_t0.nc to read these files. But there is an error: nc file does not have a aggregation dimension. How to solve the error, thanks a lot.

[AndresSepulveda]

Hi,

Probably your files do not contain a RECORD dimension, which is used to order (or join) the files. For this case, time should be the record dimension. Try changing that dimension using the NCO tools, here is avery good webpage that explains how to do this, among many other things with NCO

http://research.jisao.washington.edu/data_sets/nco/

(see example #93

[limaolin1981611]

here is the gfs and hycom files

[limaolin1981611]

20200706.zip

[limaolin1981611]

If I put the files into in a single, multiple-record NetCDF file, it works perfectly. Why and How can I run the model using multiple single-record files. Thanks a lot.

[knutfrode]

I believe you need a RECORD dimension, like Andres said above. Can you post an ncdump of one of the files?

[limaolin1981611]

I used Panoply to see the detail of the nc file.

File "gfs_1p00_sfc_wind_uv_ChinaSea_2020071400_t000.nc" File type: NetCDF-3/CDM

netcdf file:/G:/Project/SAR/hycom_gfs/20200714/gfs_1p00_sfc_wind_uv_ChinaSea_2020071400_t000.nc { dimensions: longitude = 29; latitude = 44; time = 1; variables: float wind_u10(time=1, latitude=44, longitude=29); :long_name = "10 metre wind (U10M)"; :standard_name = "x_wind"; :units = "m s-1";

float wind_v10(time=1, latitude=44, longitude=29);
  :long_name = "10 metre wind (V10M)";
  :standard_name = "y_wind";
  :units = "m s-1";

double time(time=1);
  :units = "seconds since 2000-01-01 00:00:00";
  :calendar = "standard";
  :long_name = "time";
  :standard_name = "time";
  :step = "3";
  :axis = "T";
  :_CoordinateAxisType = "Time";

float latitude(latitude=44);
  :long_name = "latitude coordinate";
  :standard_name = "latitude";
  :units = "degrees_north";
  :step = "0.08";
  :_CoordinateAxisType = "Lon";
  :axis = "Y";

float longitude(longitude=29);
  :long_name = "longitude coordinate";
  :standard_name = "longitude";
  :units = "degrees_east";
  :axis = "X";
  :_CoordinateAxisType = "Lon";

// global attributes: :_CoordSysBuilder = "ucar.nc2.dataset.conv.DefaultConvention"; }

[limaolin1981611]

File "hycom_glb_sfc_water_uv_ChinaSea_2020071400_t000.nc" File type: NetCDF-3/CDM

netcdf file:/E:/Project/SAR/hycom_gfs/20200714/hycom_glb_sfc_water_uv_ChinaSea_2020071400_t000.nc { dimensions: longitude = 375; latitude = 562; time = 1; depth = 1; variables: float water_u(time=1, depth=1, latitude=562, longitude=375); :long_name = "Sea water x velocity"; :standard_name = "x_sea_water_velocity"; :units = "m s-1"; :coordinates = "lon lat depth time";

float water_v(time=1, depth=1, latitude=562, longitude=375);
  :long_name = "Sea water y velocity";
  :standard_name = "y_sea_water_velocity";
  :units = "m s-1";
  :coordinates = "lon lat depth time";

double time(time=1);
  :units = "seconds since 2000-01-01 00:00:00";
  :calendar = "standard";
  :long_name = "time";
  :standard_name = "time";
  :step = "3";
  :axis = "T";
  :_CoordinateAxisType = "Time";

float depth(depth=1);
  :long_name = "depth";
  :standard_name = "depth";
  :units = "m";
  :axis = "Z";
  :_CoordinateAxisType = "Height";

float latitude(latitude=562);
  :long_name = "latitude coordinate";
  :standard_name = "latitude";
  :units = "degrees_north";
  :step = "0.08";
  :_CoordinateAxisType = "Lon";
  :axis = "Y";

float longitude(longitude=375);
  :long_name = "longitude coordinate";
  :standard_name = "longitude";
  :units = "degrees_east";
  :axis = "X";
  :_CoordinateAxisType = "Lon";

// global attributes: :_CoordSysBuilder = "ucar.nc2.dataset.conv.DefaultConvention"; }

[knutfrode]

Can you also please post the output from command line command $ readerinfo.py <file>

This is equivalent to >>> print(r) where r is the created rwader object.

[limaolin1981611]

How Can I use 'readerinfo.py ' in the command line

[limaolin1981611]

[limaolin1981611]

[limaolin1981611]

@knutfrode Thank a lot. I have post the output from command line command $ readerinfo.py .

[knutfrode]

Googling your error message (nc file does not have a aggregation dimension.) gives this as first result: https://stackoverflow.com/questions/20340977/using-mfdataset-to-combine-netcdf-files-in-python Indicating that in your file, time is not the slowest varying (leftmost) dimension in your files (probably this is depth instead).

So you should probably either rearrange your dimensions, or you can try to add aggdim='time' to the MFDataset constructor here: https://github.com/OpenDrift/opendrift/blob/master/opendrift/readers/reader_netCDF_CF_generic.py#L136

[limaolin1981611]

@knutfrode aggdim='time' to the MFDataset constructor，do you mean as follows:

        if ('*' in filestr) or ('?' in filestr) or ('[' in filestr):
            self.logger.info('Opening files with MFDataset')
            if has_xarray:
                self.Dataset = xr.open_mfdataset(filename)
            else:
                self.Dataset = MFDataset(filename)
                aggdim = 'time'

[knutfrode]

No, I mean self.Dataset = MFDataset(filename, aggdim='time')

[limaolin1981611]

@knutfrode Thanks a lot. It works well. There is another question. The multiple single gfs and hycom files store hourly outputs. If I add several files that store 15 minutes outputs, the time of files may be 00:00, 00:15, 00:30, 00:45, 01:00, 02:00, 02:15, 02:30, 03:00, .... Can the leeway model works well?

[knutfrode]

Yes, this is no problem.

On Fri, Jul 17, 2020, 16:24 limaolin1981611 notifications@github.com wrote:

@knutfrode https://github.com/knutfrode Thanks a lot. It works well. There is another question. The multiple single gfs and hycom files store hourly outputs. If I add several files that store 15 minutes outputs, the time of files may be 00:00, 00:15, 00:30, 00:45, 01:00, 02:00, 02:15, 02:30, 03:00, .... Can the leeway model works well?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/OpenDrift/opendrift/issues/336#issuecomment-660135169, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABH25IZPIO65LY7TQAUUKCTR4BNH7ANCNFSM4O2CF36Q .

[limaolin1981611]

@knutfrode Thanks very much.