The nansat ingestor manager seems to fail in adding metadata

[mortenwh]

I am using the new ascat mapper in https://github.com/nansencenter/nansat/commit/185ae9c2232c8c27d1869185929d7bd18e77bd39 - but the metadata is not registered when I ingest with the nansat_ingestor. I thought I had done it correctly. I'm taking holiday in 25 minutes. Would be good if @akorosov could look at it....

/home/vagrant/miniconda/lib/python2.7/site-packages/geospaas/nansat_ingestor/managers.py:76: UserWarning: entry_title is not provided in Nansat metadata!
  warnings.warn('%s is not provided in Nansat metadata!' % name)
/home/vagrant/miniconda/lib/python2.7/site-packages/geospaas/nansat_ingestor/managers.py:76: UserWarning: entry_id is not provided in Nansat metadata!
  warnings.warn('%s is not provided in Nansat metadata!' % name)
/home/vagrant/miniconda/lib/python2.7/site-packages/geospaas/nansat_ingestor/managers.py:76: UserWarning: summary is not provided in Nansat metadata!
  warnings.warn('%s is not provided in Nansat metadata!' % name)
/home/vagrant/miniconda/lib/python2.7/site-packages/geospaas/nansat_ingestor/managers.py:86: UserWarning: ISO_topic_category is not provided in Nansat metadata!
  warnings.warn('%s is not provided in Nansat metadata!' % name)
/home/vagrant/miniconda/lib/python2.7/site-packages/geospaas/nansat_ingestor/managers.py:86: UserWarning: gcmd_location is not provided in Nansat metadata!
  warnings.warn('%s is not provided in Nansat metadata!' % name)
/home/vagrant/miniconda/lib/python2.7/site-packages/geospaas/nansat_ingestor/managers.py:86: UserWarning: data_center is not provided in Nansat metadata!
  warnings.warn('%s is not provided in Nansat metadata!' % name)
/home/vagrant/miniconda/lib/python2.7/site-packages/nansat/domain.py:567: UserWarning: > 180 deg correction to longitudes - disabled..
  warnings.warn("> 180 deg correction to longitudes - disabled..")

[mortenwh]

You can try with this:

./manage.py ingest /mnt/10.11.12.232/sat_downloads/ASCAT/metop_a/12km/2010/001/ascat_20100101_024501_metopa_16614_eps_o_125_1018_ovw.l2.nc

And use the branch hotfix365_scatterometer_mappers in nansat.. (https://github.com/nansencenter/nansat/tree/hotfix365_scatterometer_mappers)

[akorosov]

I was not able to open the file with Nansat. I fixed a couple of bugs but I don't how to deal with that one:

In [1]: from nansat import Nansat

In [2]: n = Nansat('/Data/sat/downloads/ASCAT/metop_a/12km/2010/001/ascat_20100101_024501_metopa_16614_eps_o_125_1018_ovw.l2.nc')
=>MetOp-A ASCAT Level 2 12.5 km Ocean Surface Wind Vector Product<=
Warning 1: dimension #1 (NUMCELLS) is not a Longitude/X dimension.
Warning 1: dimension #0 (NUMROWS) is not a Latitude/Y dimension.
Warning 1: dimension #1 (NUMCELLS) is not a Longitude/X dimension.
Warning 1: dimension #0 (NUMROWS) is not a Latitude/Y dimension.
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
~/py/nansat/nansat/mappers/mapper_netcdf_cf.py in _band_dict(self, subfilename, band_num, subds, band, band_metadata)
    267                 band_metadata['time_iso_8601'] = self._time_count_to_np_datetime64(
--> 268                     band_metadata[timecountname])
    269             except KeyError as e:

KeyError: 'NETCDF_DIM_time'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-2-d38726b6430a> in <module>()
----> 1 n = Nansat('/Data/sat/downloads/ASCAT/metop_a/12km/2010/001/ascat_20100101_024501_metopa_16614_eps_o_125_1018_ovw.l2.nc')

~/py/nansat/nansat/nansat.py in __init__(self, filename, mapper, log_level, **kwargs)
    151         self._init_empty(filename, log_level)
    152         # Create VRT object with mapping of variables
--> 153         self.vrt = self._get_mapper(mapper, **kwargs)
    154 
    155     def __getitem__(self, band_id):

~/py/nansat/nansat/nansat.py in _get_mapper(self, mappername, **kwargs)
   1139                 # create a Mapper object and get VRT dataset from it
   1140                 try:
-> 1141                     tmp_vrt = nansatMappers[iMapper](self.filename, gdal_dataset, metadata, **kwargs)
   1142                     self.logger.info('Mapper %s - success!' % iMapper)
   1143                     self.mapper = iMapper.replace('mapper_', '')

~/py/nansat/nansat/mappers/scatterometers.py in __init__(self, filename, gdal_dataset, metadata, quartile, *args, **kwargs)
     18     def __init__(self, filename, gdal_dataset, metadata, quartile=0, *args, **kwargs):
     19 
---> 20         super(Mapper, self).__init__(filename, gdal_dataset, metadata, *args, **kwargs)
     21 
     22         intervals = [0,1,2,3]

~/py/nansat/nansat/mappers/mapper_netcdf_cf.py in __init__(self, filename, gdal_dataset, gdal_metadata, *args, **kwargs)
     64 
     65         # Add bands with metadata and corresponding values to the empty VRT
---> 66         self.create_bands(self._band_list(gdal_dataset, metadata, *args, **kwargs))
     67 
     68         # Check size?

~/py/nansat/nansat/mappers/mapper_netcdf_cf.py in _band_list(self, gdal_dataset, gdal_metadata, netcdf_dim, bands, *args, **kwargs)
    225                 # append band with src and dst dictionaries
    226                 bdict = self._band_dict(fn, band_num, subds, band=band,
--> 227                         band_metadata=band_metadata)
    228                 if bdict:
    229                     metadictlist.append(bdict)

~/py/nansat/nansat/mappers/mapper_netcdf_cf.py in _band_dict(self, subfilename, band_num, subds, band, band_metadata)
    273                 warnings.warn(
    274                         '%s: %s - %s Continuing without time metadata for band %s'
--> 275                         %(e.__repr__().split('(')[0], e.message, e.__doc__,
    276                             band_metadata['NETCDF_VARNAME']))
    277 

AttributeError: 'KeyError' object has no attribute 'message'

[mortenwh]

This is not an issue any more. The new package django-geo-spaas-scatterometry handles ingestion of scatterometer data, and works well.