search

ukaea / piezo

1 stars 0 forks source link

Consolidate spark image for both prometheus and python support #71

Closed robert-clegg-tessella closed 5 years ago

robert-clegg-tessella commented 5 years ago

The standard Spark image ("gcr.io/spark-operator/spark:v2.4.0") supports Prometheus but not Python. The CERN Spark image ("gitlab-registry.cern.ch/db/spark-service/docker-registry/spark:v2.4.0-hadoop3-0.7") is in the opposite situation. We currently have to switch between the two in the "image" field of the validation_rules.json file when deploying the web app.

We need both at the same time.

oliver-tarrant-tessella commented 5 years ago

On hold as to test need to have completed stories #23 and #4 first

oliver-tarrant-tessella commented 5 years ago

Closing as has been incorporated into #23 where a consolidated spark image is needed to contain prometheus configurations and still run jobs requiring S3 access