-
Although having priority queue on CRD alone won't make priority job run happens properly I'd like to propose the Spark Operator to support very basic priority queues:
1) priority from low -> normal…
-
### What happened?
I am trying to deploy a Beam job (Python Beam) that runs on a PortableRunner (Flink Runner) in my Kubernetes cluster.
I have not experienced issues prior with Beam using the Flin…
-
can mainApplicationFile python file can be fetched from volume mount/S3 checkout instead of local in docker images?
Im using spark opertor in kubernetes, instead of spark submit..
if it is **local:/…
-
- [x] ✋ I have searched the open/closed issues and my issue is not listed.
#### Please describe your question here
As per documentation base spark version for operator version v1beta2-1.6.x-3.5.…
-
### Issues Policy acknowledgement
- [X] I have read and agree to submit bug reports in accordance with the [issues policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md)
### Willi…
-
ETA: 2024-06-30
We want to use IPv4 addresses of SPARK nodes as the scarce resource that makes it expensive for a single party to run many nodes. ATM, we rely on the trusted spark-api service to re…
-
cc @Fokko .
This is a super simple implementation of an iceberg client for dask. It works for the limited couple of datasets I have available including
- version metadata choice
- snapshot time …
-
### Describe the bug
We easily run into this problem when running queries with `spark.comet.exec.shuffle.mode=jvm`:
```
org.apache.spark.memory.SparkOutOfMemoryError: Unable to acquire 67108848…
-
Hi,
I got a strange problem. After a couple of days(maybe 10 days to 20 days), the MutatingWebhook stopped working. The 'spark-operator' pod was still there without any restarts(I can see restarts …
-
## Description
Spark operator pod crashes with ImagePullBackOffError.
- [ X] ✋ I have searched the open/closed issues and my issue is not listed.
## Reproduction Code [Required]
Steps …