search

hyriver / hyriver.github.io

A Python software stack for retrieving hydroclimate data from web services.
https://docs.hyriver.io
Other
96 stars 12 forks source link

Error in py3dep when used in pygeoapi #1

Closed rmcd-mscb closed 3 years ago

rmcd-mscb commented 3 years ago

I'm using py3dep package as part of a process we are adding to our local copy pygeoapi. py3dep is used to pull topography from which elevation is interpolated onto a cross-section. When instantiating pygeoapi with this process included I get the following error: RuntimeError: There is no current event loop in thread 'Thread-28'. It looks like it connected to nest_asyncio.apply in pygeoogc/utils.py. The traceback follows. Any help would be greatly appreciated.

Traceback (most recent call last): File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/flask/app.py", line 2069, in call return self.wsgi_app(environ, start_response) File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/flask/app.py", line 2054, in wsgi_app response = self.handle_exception(e) File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/flask/app.py", line 2051, in wsgi_app response = self.full_dispatch_request() File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/flask/app.py", line 1501, in full_dispatch_request rv = self.handle_user_exception(e) File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/flask/app.py", line 1499, in full_dispatch_request rv = self.dispatch_request() File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/flask/app.py", line 1485, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(req.view_args) File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/pygeoapi-0.11.dev0-py3.9.egg/pygeoapi/flask_app.py", line 375, in get_processes headers, statuscode, content = api.describe_processes( File "/home/rmcd/miniconda3/envs/pygeoapidev/lib/python3.9/site-packages/pygeoapi-0.11.dev0-py3.9.egg/pygeoapi/api.py", line 122, in inner return func(cls, headers, format_, *args, *kwargs) File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/pygeoapi-0.11.dev0-py3.9.egg/pygeoapi/linked_data.py", line 54, in inner return func(args, kwargs) File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/pygeoapi-0.11.dev0-py3.9.egg/pygeoapi/api.py", line 1765, in describe_processes p = load_plugin('process', File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/pygeoapi-0.11.dev0-py3.9.egg/pygeoapi/plugin.py", line 105, in load_plugin module = importlib.import_module(packagename) File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1030, in _gcd_import

File "", line 1007, in _find_and_load

File "", line 986, in _find_and_load_unlocked

File "", line 680, in _load_unlocked

File "", line 790, in exec_module

File "", line 228, in _call_with_frames_removed

File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/pygeoapi-0.11.dev0-py3.9.egg/pygeoapi/process/nldi_xsatpoint.py", line 6, in from nldi_xstool.nldi_xstool import getXSAtPoint File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/nldi_xstool/nldi_xstool.py", line 7, in import py3dep File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/py3dep/init.py", line 6, in from .py3dep import deg2mpm, elevation_bycoords, elevation_bygrid, get_map File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/py3dep/py3dep.py", line 12, in from pygeoogc import WMS, MatchCRS, RetrySession, ServiceURL File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/pygeoogc/init.py", line 16, in from .pygeoogc import WFS, WMS, ArcGISRESTful, ServiceURL File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/pygeoogc/pygeoogc.py", line 16, in from . import utils File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/pygeoogc/utils.py", line 35, in nest_asyncio.apply() File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/site-packages/nest_asyncio.py", line 12, in apply loop = loop or asyncio.get_event_loop() File "/home/rmcd/miniconda3/envs/pygeoapi_dev/lib/python3.9/asyncio/events.py", line 642, in get_event_loop raise RuntimeError('There is no current event loop in thread %r.' RuntimeError: There is no current event loop in thread 'Thread-28'.

The function using py3dep is here: https://github.com/ACWI-SSWD/nldi_xstool/blob/8909af7907bee0965f1c848e5d26db6e644a59b7/nldi_xstool/nldi_xstool.py#L97

cheginit commented 3 years ago

Hi Richard,

The bad news is that I couldn't reproduce the error so I couldn't track it down. The good news is that in a new upcoming release I moved all the async parts of pygeoogc to a new package called async_retriever and removed all its dependencies from pygeoogc. So now you you will not face this issue. I still haven't released this new version officially but you can give it try till then by installing pygeoogc and py3dep from github like so:

pip install \
git+https://github.com/cheginit/pygeoogc.git \
git+https://github.com/cheginit/pygeoutils.git \
git+https://github.com/cheginit/py3dep.git
rmcd-mscb commented 3 years ago

Thanks Taher for the quick response. I'll give that a try and let you know how it goes. Thanks again, Rich

cheginit commented 3 years ago

Sure!

I pushed a fix to async_retriever that hopefully will fix this issue.

rmcd-mscb commented 3 years ago

@cheginit - That worked - Thank you! Just one note - it looks like orjson maybe missing from the requirements of pygeoutils. It was missing, so had to install it individually.

again - thanks Taher

cheginit commented 3 years ago

Glad it solved the issue. I removed its dependency on orjson in the new release. I will check to see if I missed something. Thanks for reporting it.

cheginit commented 3 years ago

I fixed the dependency issue.

Also, in the new version I added support for persistent caching which can significantly speed up send/recieves. So I highly recommend that you install the new optional dependency, requests-cache. You don't have to change anything in your code since py3dep automatically uses caching if it detects that requests-cache is installed.

cheginit commented 3 years ago

I am going to close this issue since it's fixed. Please feel free to re-open if needed,

rmcd-mscb commented 3 years ago

Thanks again Taher

cheginit commented 3 years ago

Sure. Pygeoapi sounds like an interesting project, let me know if there's a way that I can contribute.

cheginit commented 3 years ago

@rmcd-mscb Just wanted to let you know that I added support for all four services of PyGeoAPI to pynhd. You can find it here.