-
The 4th container mask isn't currently supported, as the property does not get mapped to a Spark configuration. Specifically:
https://github.com/jahstreet/spark-on-kubernetes-docker/blob/1dc45d3053…
-
When I run a spark job with `spark.shuffle.service.enabled=true` (without `spark.local.dir` property), I receive an exception
```
Exception in thread "main" java.lang.IllegalArgumentException: req…
-
This is a feature request to support multiple directories for spark.local.dir setup.
spark.local.dir use "/tmp" as the default setting ([link](https://spark.apache.org/docs/latest/configuration.html)…
-
### Expected Behavior
-
### Actual Behaviour
In the R Spark flow example provided here: https://kestra.io/plugins/plugin-spark/tasks/io.kestra.plugin.spark.RSubmit
What should be SPARK_HOME set as…
-
Related: https://github.com/kubeflow/katib/issues/2049
We need to improve our release process for Training Operator:
- Branch names should follow this pattern: `release-X.Y`. Similar to Katib or…
-
### Backend
VL (Velox)
### Bug description
I am using Glue + Velox, and the data is stored on S3. However, I found that a large portion of the task running time is spent on reading remote data. Aft…
-
I was chatting with @mccheah and @jlz27 about what it would take for us to run integration tests in CircleCI, and Jason has a setup that works, minus one problem.
Roughly the setup is to use miniku…
-
### Please describe your problem in detail
https://github.com/volcano-sh/volcano/blob/master/docs/design/capacity-scheduling.md
In the Story 2. article says "Administrator can create two queues wi…
-
Hi all,
I wanted to check if it is possible to define an AWS IAM role that should be attached to driver and worker pods in Spark application YAML file. As per my understanding, currently these pods…
-
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)
### Search before asking
- [X] I have searched in the [issues](http…