stackabletech / spark-k8s-operator

Operator for Apache Spark-on-Kubernetes for Stackable Data Platform
https://stackable.tech
Other
51 stars 2 forks source link

Document python requirements install will not work in offline cluster #227

Open maltesander opened 1 year ago

maltesander commented 1 year ago

We do a pip install in the requirements container https://github.com/stackabletech/spark-k8s-operator/blob/2113e1cfc5309ec0ea6594c266c0d4fa25f0abd9/rust/operator-binary/src/spark_k8s_controller.rs#L185.

We should document that if pip requirements are given, this will not work in offline mode somewhere here https://github.com/stackabletech/spark-k8s-operator/blob/main/docs/modules/spark-k8s/pages/job_dependencies.adoc#spark-native-package-coordinates-and-python-requirements