earthobservations / wetterdienst

Open weather data for humans.
https://wetterdienst.readthedocs.io/
MIT License
360 stars 55 forks source link

_gdbm.error: [Errno 11] Resource temporarily unavailable: '/home/runner/.cache/wetterdienst/dogpile/metaindex.dbm' #491

Closed amotl closed 2 years ago

amotl commented 3 years ago

Hi there,

this is just for the records. When turning on parallel testing using pytest-xdist on CI on behalf of #431, the test suite croaks [1] with:

==================================== ERRORS ====================================
______________________ ERROR collecting tests/test_api.py ______________________
tests/test_api.py:6: in <module>
    from wetterdienst import Wetterdienst
wetterdienst/__init__.py:7: in <module>
    from wetterdienst.api import Wetterdienst
wetterdienst/api.py:9: in <module>
    from wetterdienst.provider.dwd.forecast import DwdMosmixRequest
wetterdienst/provider/dwd/forecast/__init__.py:4: in <module>
    from wetterdienst.provider.dwd.forecast.api import DwdMosmixRequest, DwdMosmixValues
wetterdienst/provider/dwd/forecast/api.py:27: in <module>
    from wetterdienst.provider.dwd.forecast.access import KMLReader
wetterdienst/provider/dwd/forecast/access.py:18: in <module>
    from wetterdienst.util.cache import CacheExpiry
wetterdienst/util/cache.py:72: in <module>
    metaindex_cache = make_region(
.venv/lib/python3.9/site-packages/dogpile/cache/region.py:491: in configure
    self.backend = backend_cls(arguments or {})
.venv/lib/python3.9/site-packages/dogpile/cache/backends/file.py:160: in __init__
    self._init_dbm_file()
.venv/lib/python3.9/site-packages/dogpile/cache/backends/file.py:183: in _init_dbm_file
    fh = dbm.open(self.filename, "c")
/opt/hostedtoolcache/Python/3.9.6/x64/lib/python3.9/dbm/__init__.py:95: in open
    return mod.open(file, flag, mode)
E   _gdbm.error: [Errno 11] Resource temporarily unavailable: '/home/runner/.cache/wetterdienst/dogpile/metaindex.dbm'

The exception is the same to what has been noted at [2].

With kind regards, Andreas.

[1] https://github.com/earthobservations/wetterdienst/runs/3274094097#step:8:37 [2] https://github.com/earthobservations/wetterdienst/blob/v0.20.3/docs/usage/explorer.rst#known-bugs

amotl commented 3 years ago

This is another reason to get rid of the dbmfile-based dogpile cache completely (#243). Currently, even with the patch from #431, Wetterdienst will still use a hybrid of dogpile and FSSPEC, see https://github.com/earthobservations/wetterdienst/pull/431#issuecomment-894793001.