search

kartoza / geodata-mart

Geoprocessing as a Service
https://data.kartoza.com
2 stars 3 forks source link

Outline core application structure #3

Open zacharlie opened 2 years ago

zacharlie commented 2 years ago

Define the product and structure for development

zacharlie commented 2 years ago

The application is a django stack application which uses a celery-redis backend processing system to execute tasks.

Considering the needs of the application, an opinionated stack was configured using cookie-cutter django, which was then reconfigured into a more generic stack structure for use with docker. A docker development environment is provided.

A PostGIS database backend is used, and minio is included in the stack as an S3 storage service used by default for the media root, with white noise and django compressor being used for static asset management. The intended initial structure is outlined by the following diagram:

gdm-processing-structure-wps

GeoData Processing is not expected to be managed by the media root, and is instead configured using a custom django-storage object. Although eventually it is desired that S3 compatible storage is integrated, for the time being all storage is being managed by a shared docker volume (the app/geodata directory, mounted at /qgis).

It was initially intended that processing would be handled by WPS, but as outlined by #49 this was non-trivial to achieve, so instead the base container was changed from python to qgis, and the qgis python api is called directly by celery tasks.

gdm-processing-structure-current