too many times in Dwave burst file

[mmartini-usgs]

I am processing DWave data,

runrskcdf2nc.py (which calls cdf_to_nc) generates what should be an EPIC compliant file and that file contains time_cf, time, time_2, epic_time and epic_time2.

In that configuration, we don't need epic_time*

One could argue that going forward, we should just produce files with time only, in CF convention. The EPIC times are a pain in python, though, these are burst files (time, sample) that get rejected by xarray anyway: 'time' has more than 1-dimension and the same name as one of its dimensions ('time', 'sample'). xarray disallows such variables because they conflict with the coordinates used to label dimensions.

It's not clear how this is happening. Maybe because in cdf2nc.py both ds = utils.create_epic_times(ds) and ds = utils.create_2d_time(ds) are applied? We could comment out ds = utils.create_epic_time(ds) on line 42, but I'll bet there is too much other code dependent on that. So we may want some keyword argument to do this.

Marinna

[dnowacki-usgs]

I already have an is_cf() function which checks to see if Conventions: CF-1.6 exists in the attrs. The AQD and EXO code already uses this for straight-to-CF .nc file creation. You're right that it should be extended to Dwave data, and I may have started down that road but never finished it for some reason (maybe as you allude in your final paragraph). Definitely would welcome something that implements straight-to-CF for dwave data.

[mmartini-usgs]

is_cf() checks for CF compliance or generates CF compliance? How does that help with time variables?

It has to be dealt with on file creation, there is not easy way to delete the variables later. netCDF does not allow deletion of variables, and xarray will not open it since it has time, sample with an ordinate dimension called time (we will have to deal with that somehow, some day).

[dnowacki-usgs]

Neither, exactly. It's just a function to check whether Compliance == 'CF-1.6' which we can use as a flag to bypass things like creating epic_time, and for putting in standard_name instead of EPIC key codes.

[dnowacki-usgs]

There is still testing to be done, but I believe in the current master, if Conventions == 'CF-1.6', now will produce a file with CF-compliant times for dwave and AQD wvs files. Files still aren't fully CF compliant, but at least the times are something xarray can read. You'll see that there is an if-block that avoids running utils.create_epic_time, among other things, as you alluded to in your first post. Does this get most of the way of solving the issue?

[mmartini-usgs]

Ah - OK, got it. This is undocumented. So we set "conventions" in the global attribute file to 'CF-1.6'? Or do we set compliance='CF-1.6' when running - which script? A code search of the repository only shows 'CF-1.6' in the test files.

[mmartini-usgs]

So here's what happens when I put conventions: "CF-1.6" in the global attributes file: runrskrsk2cdf.py does fine, and outputs CF time. runrskcdf2nc.py does fine and gives me time, epic_time, epic_time2, epic_time_2d and epic_time2_2d. runrsknc2waves.py died with unable to decode time units 'msec since 0:00 GMT' with the default calendar. Try opening your dataset with decode_times=False.

[dnowacki-usgs]

Needs to be capital C: Conventions: 'CF-1.6' in the yaml config file

[dnowacki-usgs]

We have been processing CF RBR data for a while now, and the Conventions: CF-1.X is now described in the docs, so I'm going to close this. Marinna, if you're out there and still have feelings about it, please reopen 😃