search

EIDA / userfeedback

This repository is meant to collect feedback from EIDA users by means of its Issue Tracker
11 stars 5 forks source link

[Federator] Something in the Federator makes mass downloader crash #154

Closed javiquinte closed 9 months ago

javiquinte commented 10 months ago

Describe the problem Some code of us makes use of the Obspy mass_downloader. Recently, under the most normal operation (no parameters or weird configuration) it started to crash. The cause seems to be the Federator. We tried it restricting the data centers to use and the Federator is the only one raising the same error.

To Reproduce

>>> from obspy.clients.fdsn.mass_downloader.mass_downloader import MassDownloader
>>> downloader=MassDownloader(providers=["EIDA"], debug=True)
[2023-12-04 12:42:01,189] - obspy.clients.fdsn.mass_downloader - INFO: Initializing FDSN client(s) for EIDA.
Installed new opener with handlers: [<obspy.clients.fdsn.client.CustomRedirectHandler object at 0x7fb93bd9f2e0>]
Base URL: http://eida-federator.ethz.ch
Request Headers: {'User-Agent': 'ObsPy/1.4.0 (Linux-5.15.0-86-generic-x86_64-with-glibc2.34, Python 3.8.18)'}
Downloading http://eida-federator.ethz.ch/fdsnws/dataselect/1/application.wadl with requesting gzip compression
Downloading http://eida-federator.ethz.ch/fdsnws/event/1/application.wadl with requesting gzip compression
Downloading http://eida-federator.ethz.ch/fdsnws/station/1/application.wadl with requesting gzip compression
Downloading http://eida-federator.ethz.ch/fdsnws/event/1/catalogs with requesting gzip compression
Downloading http://eida-federator.ethz.ch/fdsnws/event/1/contributors with requesting gzip compression
Downloaded http://eida-federator.ethz.ch/fdsnws/dataselect/1/application.wadl with HTTP code: 200
Downloaded http://eida-federator.ethz.ch/fdsnws/event/1/catalogs with HTTP code: 200
Downloaded http://eida-federator.ethz.ch/fdsnws/event/1/contributors with HTTP code: 200
Downloaded http://eida-federator.ethz.ch/fdsnws/event/1/application.wadl with HTTP code: 200
Downloaded http://eida-federator.ethz.ch/fdsnws/station/1/application.wadl with HTTP code: 200
Discovered dataselect service
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/javier/.pyenv/versions/gmv/lib/python3.8/site-packages/obspy/clients/fdsn/mass_downloader/mass_downloader.py", line 113, in __init__
    self._initialize_clients()
  File "/home/javier/.pyenv/versions/gmv/lib/python3.8/site-packages/obspy/clients/fdsn/mass_downloader/mass_downloader.py", line 327, in _initialize_clients
    clients = p.map(_get_client, self.providers)
  File "/home/javier/.pyenv/versions/3.8.18/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/javier/.pyenv/versions/3.8.18/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
  File "/home/javier/.pyenv/versions/3.8.18/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/javier/.pyenv/versions/3.8.18/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/javier/.pyenv/versions/gmv/lib/python3.8/site-packages/obspy/clients/fdsn/mass_downloader/mass_downloader.py", line 302, in _get_client
    this_client = Client(client_name, debug=self.debug)
  File "/home/javier/.pyenv/versions/gmv/lib/python3.8/site-packages/obspy/clients/fdsn/client.py", line 277, in __init__
    self._discover_services()
  File "/home/javier/.pyenv/versions/gmv/lib/python3.8/site-packages/obspy/clients/fdsn/client.py", line 1610, in _discover_services
    self.services["event"] = WADLParser(wadl).parameters
  File "/home/javier/.pyenv/versions/gmv/lib/python3.8/site-packages/obspy/clients/fdsn/wadl_parser.py", line 35, in __init__
    url = self._xpath(doc, "/application/resources")[0].get("base")
IndexError: list index out of range

Client Python 3.8 - Obspy 1.4.0

Additional context This was done during the morning of 4th of December. To our understanding it worked before.

Jollyfant commented 10 months ago

It may be no surprise that http://eida-federator.ethz.ch/fdsnws/event/1/application.wadl cannot be parsed. 🧑‍🔬

kaestli commented 10 months ago

ok, this was a trial to show the default content http://eida-federator.ethz.ch/ (rather than 404) on more URLs where people may expect or need explanation. was a bit too greedy :-) I changed that; now everything trying to access a not existing event service will get a http 404 instead of an explanation page with http 200. It is fixed when it fixes your issue, @javiquinte ...