radanalyticsio / spark-operator

Operator for managing the Spark clusters on Kubernetes and OpenShift.
Apache License 2.0
157 stars 61 forks source link

How to add volumes or ephemeral storage to spark application configuration used by Spark Operator #342

Open krisshub opened 3 years ago

krisshub commented 3 years ago

Description:

Is there an option to add volume or increase ephemeral storage for the spark application

Steps to reproduce:

apiVersion: radanalytics.io/v1
kind: SparkApplication
metadata:
  name: xxx
  namespace: default
spec:
  type: Scala
  mode: cluster
  image: "xxx"
  imagePullPolicy: Always
  mainClass: xxx
  mainApplicationFile: "xxx"
  arguments: "xxx"
  restartPolicy: Never
  sparkVersion: "3.0.0"
  driver:
    cores: 1
    coreLimit: "1000m"
    memory: "10240m"
    labels:
      version: 3.0.0
  executor:
    cores: 2
    coreLimit: "2000m"
    memory: "12288m"
    instances: 44
    labels:
      version: 3.0.0