.. |BuildStatus| image:: https://jenkins.ethz.ch/buildStatus/icon?job=mediatorws :alt: Build Status .. _BuildStatus: https://jenkins.ethz.ch/job/mediatorws
EIDA NG Mediator/Federator webservices
.. note::
Disclaimer: This is an archived repository. Here, no further development takes place. Instead, development continues at https://github.com/EIDA/eidaws and https://github.com/EIDA/eidaws-federator-deployment.
|BuildStatus|_ (py35, py36)
This repository is intended to contain the source code for three of the web services of EIDA NG: (i) the federator, (ii) stationlite and (iii) the mediator.
Federator: Federate fdsnws-station, fdsnws-dataselect, and eidaws-wfcatalog requests across all EIDA nodes. This means, a user can issue a request against a federator endpoint without having to know where the data is hosted. In order to discover the information location the eidaws-stationlite web service is used.
StationLite: A lightweight fdsnws-station web service providing routing information. The stream epoch information is returned fully resolved (Stream epochs do not contain wildcard characters anymore.). The information location is harvested making use of eidaws-routing configuration files and fdsnws-station.
Mediator: This service allows queries across different web service domains, e.g., fdsnws-station and fdsnws-dataselect. Example: Retrieve waveform data for all stations within a lat-lon box.
Currently, we provide an alpha version of the federator service for fdsnws-station, fdsnws-dataselect and eidaws-wfcatalog.
Besides an alpha version of the stationlite service is implemented.
Installation
_Run the Test WSGI servers
_StationLite harvesting
_Logging (application level)
_Missing features and limitations
_Development <https://github.com/EIDA/mediatorws/tree/master/docs/installing/development.rst>
_Docker <https://github.com/EIDA/mediatorws/tree/master/docs/installing/docker.rst>
_The examples bellow use the built-in Flask <http://flask.pocoo.org/>
_ server,
which is not recommended to use in production. In production environments the
usage of a WSGI server should be preferred. An exemplary setup with mod_wsgi
and Apache2 is described in the section Deploying to a webserver <https://github.com/EIDA/mediatorws/tree/master/docs/installing/development.rst#Deploying to a webserver>
_. Alternatively use Gunicorn or uWSGI.
To launch a local test WSGI server (NOT for production environments) enter:
.. code::
(venv) $ eida-federator-test --tmpdir='/path/to/tmp'
For further configuration options invoke
.. code::
(venv) $ eida-federator-test -h
The service currently writes temporary files to the :code:tmpdir
. Make sure
that plenty of space is available.
To launch a local test WSGI server (NOT for production environments) enter:
.. code::
(venv) $ eida-stationlite-test URL
:code:URL
is a database url as described at the SQLAlchemy documentation <http://docs.sqlalchemy.org/en/latest/core/engines.html#database-urls>
_.
For further configuration options invoke
.. code::
(venv) $ eida-stationlite-test -h
.. note::
The EIDA Mediator webservice currently still does not provide an installation routine. However, a test server can be started as described bellow. Note, that you have to install all dependencies required manually.
Add the repository directory to your PYTHONPATH. Then, the server can be started as
.. code::
$ python -m eidangservices.mediator.server --port=5001 --tmpdir='/path/to/tmp'
The server writes temporary files to the tmpdir, so this directory will fill up.
It is recommended to purge this directory regularly, e.g., using a tool like
tmpreaper
.
The stationlite webservice data is stored in a database which periodically
must be harvested. This is done with :code:eida-stationlite-harvest
. By means
of the eidaws-routing configuration files and the fdsnws-station webservice
:code:eida-stationlite-harvest
collects and updates the database. Information
on how to use :code:eida-stationlite-harvest
is available with
.. code::
(venv) $ eida-stationlite-harvest -h
In addition the software suite contains an empty exemplary preconfigured
SQLite database (:code:db/stationlite.db.empty
) which must be filled
initially after installing the stationlite webservice. I.e.
.. code::
(venv) $ cd $PATH_INSTALLATION_DIRECTORY/mediatorws/ (venv) $ cp -v db/stationlite.db.empty db/stationlite.db (venv) $ eida-stationlite-harvest sqlite:///$(pwd)/db/stationlite.db
Note, that harvesting may take some time until completed.
.. note::
EIDA Federator and StationLite webservices only.
For debugging purposes EIDA NG webservices also provide logging facilities.
Simply configure your webservice with a logging configuration file. Use the INI
logging configuration file format <https://docs.python.org/library/logging.config.html#configuration-file-format>
.
In case initialization failed a fallback SysLogHandler <https://docs.python.org/library/logging.handlers.html#sysloghandler>
is
set up:
.. code:: python
fallback_handler = logging.handlers.SysLogHandler('/dev/log',
'local0')
fallback_handler.setLevel(logging.WARN)
fallback_formatter = logging.Formatter(
fmt=("
An exemplary logging configuration using a SysLogHandler is located at
:code:$PATH_INSTALLATION_DIRECTORY/mediatorws/config/syslog.conf
. At
:code:$PATH_INSTALLATION_DIRECTORY/mediatorws/config/logging.config
a
StreamHandler <https://docs.python.org/library/logging.handlers.html#streamhandler>
_ is
configured.
.. note::
In order to keep the WSGI application portable you should avoid setting up
a logger writing to :code:sys.stdout
. See also:
http://modwsgi.readthedocs.io/en/develop/user-guides/debugging-techniques.html
When using an EIDA NG multithreaded webservice together with a mod_wsgi
configuration processes logging to a single file <https://docs.python.org/howto/logging-cookbook.html#logging-to-a-single-file-from-multiple-processes>
is not supported. Instead initialize your logger with a handler which
guarantees log messages to be serialized (e.g. SysLogHandler
,
SocketHandler <https://docs.python.org/library/logging.handlers.html#sockethandler>
_).
format={post,get}
)