cedadev / ceda-di

Other
2 stars 2 forks source link

multiple features in one JSON file #42

Open alaniwi opened 4 years ago

alaniwi commented 4 years ago

JSON file has features (list of dictionaries) at first level and the spatial is inside each of these...

/badc/eufar/data/aircraft/bas-twinotter/EUFAR17_86-2017_187_Bavarian_Forest/json_files/json_files/camera_jsons/EUFAR17_86-187-camera-20171013_flight_info.json
Failure in process_file for /badc/eufar/data/aircraft/bas-twinotter/EUFAR17_86-2017_187_Bavarian_Forest/json_files/json_files/camera_jsons/EUFAR17_86-187-camera-20171013_flight_info.json
Process Process-13178:
Traceback (most recent call last):
  File "/apps/contrib/jaspy/miniconda_envs/jaspy3.7/m3-4.5.11/envs/jaspy3.7-m3-4.5.11-r20181219/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
    self.run()
  File "/apps/contrib/jaspy/miniconda_envs/jaspy3.7/m3-4.5.11/envs/jaspy3.7-m3-4.5.11-r20181219/lib/python3.7/multiprocessing/process.py", line 99, in run
    self._target(*self._args, **self._kwargs)
  File "/tmp/ceda-di/python/src/ceda_di/extract.py", line 174, in process_file
    self.index_properties(filename, hand)
  File "/tmp/ceda-di/python/src/ceda_di/extract.py", line 189, in index_properties
    raise e
  File "/tmp/ceda-di/python/src/ceda_di/extract.py", line 186, in index_properties
    props = handler.get_properties()
  File "/tmp/ceda-di/python/src/ceda_di/providers/flights/generic_flight.py", line 101, in get_properties
    geospatial = self.get_geospatial()
  File "/tmp/ceda-di/python/src/ceda_di/providers/flights/generic_flight.py", line 51, in get_geospatial
    return self.content["spatial"]
KeyError: 'spatial'

Reproduce: python /tmp/ceda-di/python/src/di.py extract --no-create-files --config /tmp/ceda-di/python/config/ceda-di-eufar.json --send-to-index /badc/eufar/data/aircraft/bas-twinotter/EUFAR17_86-2017_187_Bavarian_Forest/json_files/json_files/

Cause: features element

may have to change get_properties to be a generator (or return list) and change calling code accordingly