-
@devstein We are experimenting the connection from EC2 instance (where Jupiter is setup with sparkmagic) to EMR spark cluster(Livy enabled). I am following this blog post https://towardsdatascience.co…
-
Currently Spark has it's own script for launching EC2 instances: https://spark.apache.org/docs/0.8.1/ec2-scripts.html and [source](https://github.com/apache/spark/blob/master/ec2/spark_ec2.py).
I fil…
-
I have verified the redis instance is up on aws:
```
docker run --rm -it redis redis-cli -h "ec2-3-217-106-135.compute-1.amazonaws.com" -p myPort --pass myPass
Warning: Using a passwo…
-
As of yesterday, thunder-ec2 crashes when launching a cluster with the error below. When logging into the master of the incompletely-launched cluster, /root/spark is largely empty with just an incomp…
-
If I don't specify "--spark-version", the "/root/spark/" directory contains nothing but "/conf/".
When I specify "--spark-version=2.0.0", "--spark-version=2.0.2", "--spark-version=2.1.0", or "--spa…
-
Specifically, I get these messages during launch of the cluster, and these files are indeed not in place once the cluster starts up:
```
./spark-ec2/spark-standalone/setup.sh: line 22: /root/spark/bi…
-
We should handle the cases when users do not have CSP CLI installed (eg `aws`, `az`, `gcloud`). The existing CLI calls are mainly for pulling instance type descriptions. We can cache these information…
-
Calling `thunder-ec2 start` gives me
```
Starting slaves...
Starting master...
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/thunder_python-0.5.0_dev-py2.7.egg/thu…
-
**Describe the bug**
When using Apache Spark on Amazon EMR, you have the ability to replace Apache Hive with AWS Glue as the Hive-compatible metastore, with the underlying data being stored in S3. Wh…
-
Hi all
I am trying to launch an ec2 cluster using a more up to date AMI:ami-c928c1a9
Here's my command
root@9f2c58d4fbe6:/spark-ec2# ./spark-ec2 -k ec2AccessKey -i ec2AccessKey.pem -s 2 --ami=a…