-
I changed the base image from gcr.io spark:3.1.1 to spark:3.5.0 and built my dockerfile from there where it has my DataAnalyticsReporting.jar
This is the error when the spark-operator tries to star…
-
During a recent DynamoDB migration, $SPARK_HOME/work on the worker nodes grew to 479 GB, which filled up a 485 GB disk.
To try to work around this, we set:
```
export SPARK_WORKER_OPTS='-Dspark…
-
# Error when using spark_apply method
I am using Spark Connect to perform operations with tables hosted in Unity Catalog (Databricks). When I want to use the `spark_apply` method to process them I…
-
I encounterd an exception as following when I tried to send Spark's lineage message to Kafka cluster.
```
24/07/29 22:56:26 ERROR org.apache.spark.util.Utils: throw uncaught fatal error in thread sp…
-
### What would you like to be added?
Support adding label based indexes to apiserver cacher to speed up list and watch requests by labels.
### Why is this needed?
Background: Each job of the big d…
-
I am trying to use SparkMeasure on Databricks, but unfortunately, it is not working when the Cluster is on Unity Catalog (Runtime 14.3 LTS).
When running the following code:
```
from sparkmeasu…
-
Hi there.
i meet serious problem now
I installed with following commands:
```bash
helm repo add spark-operator https://kubeflow.github.io/spark-operator
helm repo update
helm install spar…
-
Tips before filing an issue
- Have you gone through our [FAQs](https://hudi.apache.org/learn/faq/)?
- Join the mailing list to engage in conversations and get faster support at dev-subscribe@hudi.apa…
-
Upon reviewing the source code, it is evident that the ConsistentBucketClusteringExecutionStrategy is only implemented for the Spark engine.
-
The example that i am using is under
"hadoop tables" on https://cloud.google.com/dataproc-metastore/docs/apache-iceberg
`df.write.format("iceberg").mode("overwrite").save("gs://blahblah/iceberg_te…