hourly time series product

[diodon]

This code generates an hourly average/median of all variables available form all instruments from one site. It will produce an indexed ragged arrays netCDF files with all the variables and ancillary statistical variables from the binning process.

The binning method depends on the parameter an it is specified in the binningMethod.json dictionary. The variable and global metadata is obtained from the hourlyTS_metadata.json dictionary

[codecov[bot]]

Codecov Report

Merging #60 into master will decrease coverage by 34.16%. The diff coverage is 0%.

@@             Coverage Diff             @@
##           master      #60       +/-   ##
===========================================
- Coverage   89.44%   55.27%   -34.17%     
===========================================
  Files           4        5        +1     
  Lines         398      644      +246     
  Branches       63      106       +43     
===========================================
  Hits          356      356               
- Misses         26      272      +246     
  Partials       16       16

Impacted Files	Coverage Δ
aodntools/timeseries_products/hourly_timeseries.py	0% <0%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update d8662b3...f86662f. Read the comment docs.

[mhidas]

I have successfully run the code. Some comments on the output file created:

• There's a variable OBSERVATION(OBSERVATION), which is meaningless. It should only be a dimension.
• Variables instrument_index, instrument_id, and source_filehave no attributes (they should be the same as in the other product).
• Variables TIME, LATITUDE, LONGITUDE and NOMINAL_DEPTH have no attributes (except TIME also has units and calendar). They do have a _FillValue attribute, but these should not be there (these variables shouldn't have fill values).
• DEPTH and associated stats variables have _FillValue = NaNf. Should be 999999.0, like everything else.
• The stats variables (min, max, std, count) should only copy attributes units, and standard_name where present, from the main variable.
• For the _count variables,
  • units should always be "1";
  • standard_name (if the main var has one) should be "<main_var_standard_name> number_of_observations"
• Timestamps are on the half-hour, not on the hour e.g. TIME = "2018-12-13 08:30", "2018-12-13 09:30", "2018-12-13 10:30", etc...

[diodon]

Please rename the python file to hourly_timeseries.py (with underscore in the middle, not hyphen).

renamed

[diodon]

- There's a variable OBSERVATION(OBSERVATION), which is meaningless. It should only be a dimension.
- Variables instrument_index, instrument_id, and source_filehave no attributes (they should be the same as in the other product).
- Variables TIME, LATITUDE, LONGITUDE and NOMINAL_DEPTH have no attributes (except TIME also has units and calendar). They do have a _FillValue attribute, but these should not be there (these variables shouldn't have fill values).
- DEPTH and associated stats variables have _FillValue = NaNf. Should be 999999.0, like everything else.
- The stats variables (min, max, std, count) should only copy attributes units, and standard_name where present, from the main variable.
- For the _count variables,
units should always be "1";
standard_name (if the main var has one) should be "<main_var_standard_name> number_of_observations"
- Timestamps are on the half-hour, not on the hour e.g. TIME = "2018-12-13 08:30", "2018-12-13 09:30", "2018-12-13 10:30", etc...

all fixed.

[diodon]

I've decided to change the try/except clauses back to if/else, as the response is not an error in the file per se, it is a condition (e.g. missing some attribute) that could be expected to occur, ergo, easy to capture.

[mhidas]

@diodon I've just added three commits to make this work a bit better in the package, and to add unittests. Feel free to take a look and comment, then we can merge this.