-
### What happened?
I am trying to deploy a Beam job (Python Beam) that runs on a PortableRunner (Flink Runner) in my Kubernetes cluster.
I have not experienced issues prior with Beam using the Flin…
-
Trying to run a spark operator, i am using the `pi.py` file and `spark-py-pi.yaml` files
```
import sys
from random import random
from operator import add
from pyspark.sql import SparkSession…
-
### Issues Policy acknowledgement
- [X] I have read and agree to submit bug reports in accordance with the [issues policy](https://www.github.com/mlflow/mlflow/blob/master/ISSUE_POLICY.md)
### Willi…
-
If we use the `prometheus operator`, we can easily configure the target pod we want to collect metrics for using `pod monitor` crd and label selector and deploy the `prometheus server`.
The approxima…
-
In order to deploy PVC with every spark executor, it is needed to use multiple configurations mentioned in spark documentation -
From spark documentation, these are the needed configurations to use…
-
## Description
Unable to Start spark job in kubenetes
- [*] ✋ I have searched the open/closed issues and my issue is not listed.
## Reproduction Code [Required]
Steps to reproduce the be…
-
Hi,
we have 10 scheduled spark operators running every 10 minutes in the same namespace. Sometimes one of them gets stuck, we can see that operator is cycling through executors as they crash on st…
-
I have a problem changing the flink application in my pipeline. The problem is this: for example, I want to change the amount of memory for the task мanager in one of the job clusters. I change the d…
-
### Description
We are replacing presto in the system with velox, but we have done a lot of self-research and optimization on performance based on presto. We found that there is a performance gap b…
-
ETA: 2024-06-30
We want to use IPv4 addresses of SPARK nodes as the scarce resource that makes it expensive for a single party to run many nodes. ATM, we rely on the trusted spark-api service to re…