-
Describe the problem you faced
We encountered Errors in the production pipeline yesterday when we increased our EC2 instance type from c7g.4xlarge to c7g.8xlarge
To Reproduce
Steps to reprodu…
-
The path to various bash scrips seems to be incorrect in both spark-ec2.py and /root/spark-ec2/spark-standalone/setup.sh. It's easy to fix the first one; not sure about the second.
-
Is Spark 2.1 supported? It is not in the VALID_SPARK_VERSIONS in branch-2.0. What changes are needed for spark_ec2.py to support Spark 2.1?
-
Can the h2o-3/ec2 scripts be adapted to allow spot instances (similar to the spark-ec2 script).
-
**Describe the bug**
I am running standalone spark version 2.3 in an ec2-instance. And i have Hive standalone on the same instance, ranger-hive-plugin is setup and policies are working fine with hive…
-
It looks like spark 2.2.0 is officially released. Is it going to be supported in spark-ec2 shortly?
-
Originally asked at https://stackoverflow.com/questions/76392397/got-org-apache-kafka-common-errors-timeoutexception-topic-xxx-not-present-in-m
I am wondering could it be related to this library th…
-
I'm surprised that I wasn't able to find `spark-submit` anywhere on the master.
What are other folks doing to submit to Spark when using spark-ec2? Using an external system with it's own Spark pac…
-
## I'm using a standalone spark cluster:
- 3 AWS m4.xlarge instances (1 master+driver, 2 workers)
- All have java 1.8.0_121 and spark-1.6.0-bin-hadoop2.6
- I start master and join workers manually.…
-
**Describe the problem you faced**
I am facing issues for upsert operation in hudi 0.14 RLI in EMR 6.15 Spark 3.4.1 using "Record level Index".
i see insert mode working as expected but upsert ope…