-
I have a problem changing the flink application in my pipeline. The problem is this: for example, I want to change the amount of memory for the task мanager in one of the job clusters. I change the d…
-
See discussion in #719 with this last open issue. Moving to a separate issue for clarity.
> > How can you launch a spark application and then reliably wait for it to finish?
>
> I would explore…
-
### Description
We are replacing presto in the system with velox, but we have done a lot of self-research and optimization on performance based on presto. We found that there is a performance gap b…
-
Can someone help me find the code for image - gcr.io/spark-operator/spark:v3.1.1?
-
ETA: 2024-06-30
We want to use IPv4 addresses of SPARK nodes as the scarce resource that makes it expensive for a single party to run many nodes. ATM, we rely on the trusted spark-api service to re…
-
cc @Fokko .
This is a super simple implementation of an iceberg client for dask. It works for the limited couple of datasets I have available including
- version metadata choice
- snapshot time …
-
Operator deployed using:
```
apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
name: google-spark-operator
namespace: kube-system
spec:
chart:
repository: https://googlecl…
-
k8s: GKE 1.17 with auto-scaling node-pool with taint
spark-operator: v1beta2-1.2.0-3.0.0
SparkApp:
dynamicAllocation: enabled
tolerations and nodeSelector: specified
Sometimes executors get…
-
### Backend
VL (Velox)
### Bug description
### Exception information
ERROR
```
Context: Split [Hive: hdfs://xxx/xxx.parquet 4 - 133633304] Task Gluten_Stage_2_TID_2_VTID_2
Additional Cont…
-
Hi,
I'm adding the operator and sparkapplication helm charts to my auto deploy script, among other components (e.g. tomcat, zookeeper, etc). Since sparkapplication crds are defined by the operator,…