-
## Description
When submitting an application with in `sparkConf` a value for `spark.executor.pyspark.memory`, this memory is added to the `request` of the executor, but it's not included in the `y…
-
### What is the problem the feature request solves?
I'm running the 1TB TPCDS benchmark over Comet and Vanilla Spark.
I'm running on a 48Core 186G RAM machine
Here's my config:
```
/localhdd/…
-
When I try to follow the instructions at https://github.com/datarevenue-berlin/OpenMLOps/blob/master/tutorials/set-up-open-source-production-mlops-architecture-aws.md
I got to the step running: `k…
-
I am using gcr.io/spark-operator/spark:v3.0.0-gcs-prometheus and running my job in GKE cluster. When I upgraded scala 2.12, the job started giving me ClasscastException about version incompatibility b…
-
This was working fine for me till 2 days back. We updated our AKS version to 1.20 from 1.19 and I believe the spark-operator got updated to the latest image (gcr.io/spark-operator/spark-operator:lates…
-
Refer to #2 for a quick summary on Spark and RDDs. Note that RDDs are parameterized on a single type. You can have an `RDD` or an `RDD` but not an `RDD` or `RDD`. Consequently, the functions used to j…
-
When I try to start spark-operator with enabled webhook I receive that log
`I0907 10:26:15.465503 11 main.go:177] Enabling metrics collecting and exporting to Prometheus`
`I0907 10:26:15.465675…
-
# Description
Propose further work that I'd like to perform regarding the creation reusable logical relations. Also helps with identifying relations we would need with substrait.
**Delta Find Fil…
-
Hi @liyinan926, I am creating this issue to track any open items blocking merging the `multi-version-support` branch back into master.
The unit-tests were fixed as part of https://github.com/Goog…
-
Hi everyone, I have 2 spark applications after deploy operator doesn't create service for one of them. If I open logs operator I see that one application doesn't have webUIServiceName
```{
"spar…