FionaReader fails on Natural Earth 10m rivers for North America.

juseg commented 1 year ago

Description

FionaReader fails to open Natural Earth 10m rivers for North America. There seems to be an empty record which raises an AttributeError. BasicReader provides a record with geometry None, allowing e.g. for rec in shp.records(). Is this a data bug or a cartopy bug?

Code to reproduce


import cartopy

filename = cartopy.io.shapereader.natural_earth(
    resolution='10m', category='physical', name='rivers_north_america')
basic = cartopy.io.shapereader.BasicReader(filename).records()
fiona = cartopy.io.shapereader.FionaReader(filename).records()

while True:
    basic_record = next(basic)  # yields a record with geometry None
    fiona_record = next(fiona)  # raises AttributeError

Traceback

Traceback (most recent call last):
  File "/home/julien/git/code/hyoga/fionabug.py", line 12, in <module>
    fiona_record = next(fiona)
  File "/usr/lib/python3.10/site-packages/cartopy/io/shapereader.py", line 244, in records
    yield FionaRecord(item['geometry'],
  File "/usr/lib/python3.10/site-packages/cartopy/io/shapereader.py", line 116, in __init__
    self._bounds = geometry.bounds
AttributeError: 'NoneType' object has no attribute 'bounds

Full environment definition

### Operating system Manjaro Linux ### Relevant packages ``` Package Version ----------------------------- ----------- Cartopy 0.21.0 Fiona 1.8.21 pyshp 2.3.0 ``` ### Full pip list ``` Package Version ----------------------------- ----------- absl-py 1.2.0 absplots 0.2.0 affine 2.3.1 alabaster 0.7.12 apipkg 3.0.1 appdirs 1.4.4 apptools 5.2.0 arandr 0.1.10 astroid 2.12.7 asttokens 2.0.8 atpublic 3.1.1 attrs 22.1.0 autocommand 2.2.1 autopep8 1.7.0 Babel 2.10.3 backcall 0.2.0 beautifulsoup4 4.11.1 Bottleneck 1.3.5 btrfsutil 5.19 build 0.8.0 CacheControl 0.12.11 Cartopy 0.21.0 certifi 2022.6.15 cf-xarray 0.7.4 cffi 1.15.1 cftime 1.6.1 chardet 5.0.0 click 8.1.3 click-plugins 1.1.1 cligj 0.7.2 cloudpickle 2.1.0 colorama 0.4.5 commonmark 0.9.1 configobj 5.1.0.dev0 coverage 6.2 cryptography 37.0.4 cycler 0.11.0 Cython 0.29.32 dask 2022.7.1 decorator 5.1.1 dill 0.3.4 distlib 0.3.6 distributed 2022.7.1 distro 1.7.0 docopt 0.6.2 docutils 0.17.1 entrypoints 0.4 envisage 6.0.1 executing 1.0.0 Fiona 1.8.21 flake8 5.0.0 flufl.lock 7.1.1 fonttools 4.37.1 fsspec 2022.5.0 future 0.18.2 GDAL 3.5.1 greenlet 1.1.3 HeapDict 1.0.1 idna 3.3 imagesize 1.4.1 importlib-resources 5.9.0 inflect 6.0.0 iniconfig 1.1.1 installer 0.5.1 ipython 8.4.0 isort 5.10.1 jaraco.context 4.1.2 jaraco.functools 3.5.1 jaraco.text 3.9.1 jedi 0.18.1 Jinja2 3.1.2 joblib 1.1.0 kiwisolver 1.4.4 lazy-object-proxy 1.7.1 locket 1.0.0 MarkupSafe 2.1.1 matplotlib 3.5.2 matplotlib-inline 0.1.6 mayavi 4.8.0 mccabe 0.7.0 more-itertools 8.13.0 msgpack 1.0.4 munch 2.5.0 netCDF4 1.6.0 npyscreen 4.10.5 nspektr 0.4.0 numpy 1.23.2 ordered-set 4.1.0 OWSLib 0.26.0 packaging 21.3 pacman-mirrors 4.23.2 pandas 1.4.3 parso 0.8.3 partd 1.2.0 pep517 0.13.0 pexpect 4.8.0 pickleshare 0.7.5 Pillow 9.2.0 Pint 0.19.2 pip 22.2.2 platformdirs 2.5.2 pluggy 1.0.0 ply 3.11 prompt-toolkit 3.0.30 psutil 5.9.1 psycopg2 2.9.3 ptyprocess 0.7.0 pure-eval 0.2.2 py 1.11.0 pybind11 2.10.0 pycairo 1.21.0 pycodestyle 2.9.1 pycparser 2.21 pydantic 1.10.2 pydata-sphinx-theme 0.8.1 pydocstyle 6.1.1 pyface 7.4.2 pyflakes 2.5.0 Pygments 2.13.0 PyGObject 3.42.2 pylint 2.14.5 pynvim 0.4.3 pyOpenSSL 22.0.0 pyparsing 3.0.9 pyproj 3.3.1 PyQt5 5.15.7 PyQt5-sip 12.11.0 pyshp 2.3.0 pytest 7.1.3 pytest-cov 3.0.0 pytest-mpl 0.16.1 python-dateutil 2.8.2 python-lsp-jsonrpc 1.0.0 python-lsp-server 1.5.0 python-xlib 0.31 pytz 2022.1 PyYAML 6.0 QScintilla 2.13.3 ranger-fm 1.9.3 rasterio 1.3.2 requests 2.28.1 resolvelib 0.8.1 rich 12.5.0 rioxarray 0.12.1 rope 0.23.0 scipy 1.9.1 seaborn 0.11.2 setuptools 62.3.4 setuptools-scm 7.0.5 Shapely 1.8.2 six 1.16.0 snowballstemmer 2.2.0 snuggs 1.4.7 sortedcontainers 2.4.0 soupsieve 2.3.2.post1 speedtest-cli 2.1.3 Sphinx 4.5.0 sphinx-autosummary-accessors 2022.4.0 sphinx-basic-ng 0.0.1a12 sphinx-book-theme 0.3.3 sphinx-gallery 0.11.1 sphinx-rtd-theme 1.0.0 sphinxcontrib-applehelp 1.0.2 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.5 stack-data 0.5.0 TBB 0.2 tblib 1.7.0 team 1.0 tenacity 8.0.1 toml 0.10.2 tomli 2.0.1 tomlkit 0.11.4 toolz 0.12.0 tornado 6.2 traitlets 5.3.0 traits 6.3.2 traitsui 7.4.0 trove-classifiers 2022.8.31 typing_extensions 4.3.0 ueberzug 18.1.9 ufw 0.36.1 ujson 5.4.0 urllib3 1.26.12 validate 5.1.0.dev0 validate-pyproject 0.10.1 wcwidth 0.2.5 webencodings 0.5.1 websockets 10.3 whatthepatch 1.0.2 wheel 0.37.1 wrapt 1.14.1 xarray 2022.6.0 yapf 0.32.0 zict 2.2.0 zipp 3.8. ```

dopplershift commented 1 year ago

I'm not sure whose fault it is (if this is a recent problem caused by a data update), but I think we could probably work around it easily enough.

lgolston commented 1 year ago

The problem does seem to be in class FionaRecord, since the line self._bounds = geometry.bounds does not check for situations like geometry being None. I don't know the correct checks to do, but simply if geometry is not None: self._bounds = geometry.bounds prevents the error in this case.

greglucas commented 1 year ago

@lgolston, PRs with a suggested fix/workaround would be welcome!

greglucas commented 1 year ago

This was fixed by #2188

SciTools / cartopy