-
This is yaml of my spark job:
`kind: SparkApplication
metadata:
name: operatordc1
namespace: spark
spec:
type: Java
mode: cluster
image: "xiotxpcdevcr.azurecr.io/spark-custom:release…
-
## Spark 예제 앱 이미지 생성
- download spark
```
wget https://mirror.navercorp.com/apache/spark/spark-3.1.2/spark-3.1.2-bin-hadoop3.2.tgz
tar xvzf spark-3.1.2-bin-hadoop3.2.tgz
```
- set env
```
cd …
-
## Problem
### How much memory does a spark-dependencies job take while handling about 12Gb data index?
I am totally new to the spark project and I have tried serval times to run a spark-depend…
-
Run integration-test with the following command:
build/mvn integration-test -Pkubernetes -Pkubernetes-integration-tests -pl resource-managers/kubernetes/integration-tests -am
And get the following…
-
Hey there!
We are trying to experiment Spark on Ray with RAPIDS but not sure if Spark on Ray can support this case.
Here is the example Dockerfile for spark-rapids k8s setup:
https://nvidia.g…
-
When istio is set to auto inject the executors fail to talk to the driver and jobs never finish. Please add first-class support for istio.
-
## Reproduce
1. Deploy microk8s using charm + grafana agent
2. Deploy COS
3. Deploy prometheus gateway + cos-configuration-k8s
4. Run a job using spark-submit
```
spark-client.spark-submit --dep…
-
## Description
We are using spark operator v1beta2-1.6.2-3.5.0 in production. We have a spark application with the below policy.
```
restartPolicy:
onFailureRetryInterval: 10
onSubm…
-
Background:
For now, the spark job is set to run on top of Kubernetes. Python reads about 35 images locally, each image get processed using Tesseract using Spark, the result is then collected in a py…
-
### Tell us about your request
- Add Karpenter support to work with custom schedulers (e.g., Apache Yunikorn, Volcano)
- As per my understanding, Karpenter works only with the default scheduler…