search

aodn / python-aodntools

Repository for templates and code relating to generating standard NetCDF files for the Australia Ocean Data Network
GNU Lesser General Public License v3.0
10 stars 3 forks source link

Unhandled exceptions when all input files rejected #123

Closed mhidas closed 4 years ago

mhidas commented 4 years ago

When all the input files fail the initial checks, there are none left to be processed. The code doesn't catch this special case, leading to unhandled exceptions further along...

e.g. in aggregated_timeseries:

Traceback (most recent call last):
 ...
  File "/mnt/ebs/pipeline/lib/python3.5/site-packages/aodndata/moorings/products_handler.py", line 267, in _make_aggregated_timeseries
    **self.product_common_kwargs)
  File "/mnt/ebs/pipeline/lib/python3.5/site-packages/aodntools/timeseries_products/aggregated_timeseries.py", line 303, in main_aggregator
    INSTRUMENT = ds.createDimension('INSTRUMENT', size=n_files)
  File "netCDF4/_netCDF4.pyx", line 2562, in netCDF4._netCDF4.Dataset.createDimension
  File "netCDF4/_netCDF4.pyx", line 3370, in netCDF4._netCDF4.Dimension.__init__
  File "netCDF4/_netCDF4.pyx", line 1887, in netCDF4._netCDF4._ensure_nc_success
RuntimeError: NetCDF: NC_UNLIMITED size already in use

In hourly code:

Traceback (most recent call last):
...
  File "/mnt/ebs/pipeline/lib/python3.5/site-packages/aodndata/moorings/products_handler.py", line 300, in _make_hourly_timeseries
    **self.product_common_kwargs)
  File "/mnt/ebs/pipeline/lib/python3.5/site-packages/aodntools/timeseries_products/hourly_timeseries.py", line 551, in hourly_aggregator
    nc_aggregated.attrs = set_globalattr(nc_aggregated, TEMPLATE_JSON, site_code, add_attribute, parameter_names)
UnboundLocalError: local variable 'parameter_names' referenced before assignment

In velocity aggregated:

Traceback (most recent call last):
  ...
  File "/mnt/ebs/pipeline/lib/python3.5/site-packages/aodndata/moorings/products_handler.py", line 282, in _make_velocity_aggregated_timeseries
    product_url, errors = velocity_aggregated(input_list, self.product_site_code, **self.product_common_kwargs)
  File "/mnt/ebs/pipeline/lib/python3.5/site-packages/aodntools/timeseries_products/velocity_aggregated_timeseries.py", line 154, in velocity_aggregated
    INSTRUMENT = ds.createDimension('INSTRUMENT', size=n_files)
  File "netCDF4/_netCDF4.pyx", line 2562, in netCDF4._netCDF4.Dataset.createDimension
  File "netCDF4/_netCDF4.pyx", line 3370, in netCDF4._netCDF4.Dimension.__init__
  File "netCDF4/_netCDF4.pyx", line 1887, in netCDF4._netCDF4._ensure_nc_success
RuntimeError: NetCDF: NC_UNLIMITED size already in use