Support syncing PySpark applications from Git

[sbernauer]

In case you have multiple Python files that form an application it is inconvenient to put everything in a ConfigMap. It would be nice if you could use the same mechanism as implemented in Airflow using gitsync.

Related to https://github.com/stackabletech/opa-operator/issues/504, which implements the same thing for OPA regorules.

### Tasks
- [ ] Try to move commons code  into operator-rs

[iscsrwm]

As a workaround you can use an initContainer in your podOverrides.spec to sync your code. Here is an example:

driver: podOverrides: spec: initContainers:

• image: k8s.gcr.io/git-sync:v3.0.1 name: git-data volumeMounts:
  • name: git-source mountPath: /tmp/git env:
  • name: GIT_SYNC_ONE_TIME value: "true"
  • name: GIT_SYNC_REPO value: https://gitlab.com/mycompany/myapp.git
  • name: GIT_SYNC_BRANCH value: test
  • name: GIT_SYNC_DEPTH value: "1"
  • name: GIT_SYNC_DEST value: my_app
  • name: GIT_SYNC_USERNAME valueFrom: secretKeyRef: name: gitpuller-creds key: user
  • name: GIT_SYNC_PASSWORD valueFrom: secretKeyRef: name: gitpuller-creds key: password volumes:
• name: git-source emptyDir: {} replicas: 1 config: resources: cpu: min: "1" max: "4" memory: limit: "2Gi" volumeMounts:
  • name: git-source mountPath: /common

In this example your code would be available in your driver container at /common/my_app