-
This is probably mostly the same as https://github.com/GoogleCloudPlatform/spark-on-k8s-operator/issues/594 which has since been marked as closed, however, the solution described there doesn't seem to…
-
Trying to run a spark operator, i am using the `pi.py` file and `spark-py-pi.yaml` files
```
import sys
from random import random
from operator import add
from pyspark.sql import SparkSession…
-
### What happened?
I am trying to deploy a Beam job (Python Beam) that runs on a PortableRunner (Flink Runner) in my Kubernetes cluster.
I have not experienced issues prior with Beam using the Flin…
-
If we use the `prometheus operator`, we can easily configure the target pod we want to collect metrics for using `pod monitor` crd and label selector and deploy the `prometheus server`.
The approxima…
-
In order to deploy PVC with every spark executor, it is needed to use multiple configurations mentioned in spark documentation -
From spark documentation, these are the needed configurations to use…
-
Platform: `minikube version: v1.24.0`
tl;dr airflow won't start, logs of everything are listed below
I'm trying to recreate everything and I'm stuck with this part. I've been waiting for some t…
-
## Description
Unable to Start spark job in kubenetes
- [*] ✋ I have searched the open/closed issues and my issue is not listed.
## Reproduction Code [Required]
Steps to reproduce the be…
-
I have a problem changing the flink application in my pipeline. The problem is this: for example, I want to change the amount of memory for the task мanager in one of the job clusters. I change the d…
-
### Description of the problem / feature request:
Bazel query today operates at a target level. It helpfully also includes generator_name, generator_function, and generator_location attributes for…
-
Hi,
we have 10 scheduled spark operators running every 10 minutes in the same namespace. Sometimes one of them gets stuck, we can see that operator is cycling through executors as they crash on st…