Open-EO / openeo-geopyspark-driver

OpenEO driver for GeoPySpark (Geotrellis)
Apache License 2.0
26 stars 5 forks source link

OpenEO Geopyspark Driver

[Status]()

Python version: at least 3.8

This driver implements the GeoPySpark/Geotrellis specific backend for OpenEO.

It does this by implementing a direct (non-REST) version of the OpenEO client API on top of GeoPySpark.

A REST service based on Flask translates incoming calls to this local API.

Technology stack

Operating environment dependencies

This backend has been tested with:

Public endpoint

https://openeo.vito.be/openeo/

Running locally

Set up your (virtual) environment with necessary dependencies:

# Install Python package and its depdendencies
pip install . --extra-index-url https://artifactory.vgt.vito.be/artifactory/api/pypi/python-openeo/simple

# Get necessary JAR dependency files for running Geopyspark driver
python scripts/get-jars.py

For development, refer to docs/development for more information. You can run the service with:

export SPARK_HOME=$(find_spark_home.py)
export HADOOP_CONF_DIR=/etc/hadoop/conf
export FLASK_DEBUG=1
python openeogeotrellis/deploy/local.py

For production, a gunicorn server script is available: PYTHONPATH=. python openeogeotrellis/server.py

Running locally with Docker

This will set up the environment (Java/Python with dependencies) and makes it possible to run simple process graphs. For more information, see: docker/local_batch_job/

Running on the Proba-V MEP

The web application can be deployed by running: sh scripts/submit.sh This will package the application and it's dependencies from source, and submit it on the cluster. The application will register itself with an NginX reverse proxy using Zookeeper.