search

nchammas / flintrock

A command-line tool for launching Apache Spark clusters.
Apache License 2.0
636 stars 116 forks source link

Spark install fails intermittently #253

Closed ghost closed 6 years ago

ghost commented 6 years ago

After launching/destroying a handful (~5) clusters, new launches fail with the error below. When I ssh into the driver node I notice that spark-2.3.0-bin-hadoop2.6.tgz is something like 300 bytes.

[54.174.89.133] Installing Spark... from: https://s3.amazonaws.com/spark-related-packages/spark-2.3.0-bin-hadoop2.6.tgz % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 300 0 300 0 0 893 0 --:--:-- --:--:-- --:--:-- 895

gzip: spark-2.3.0-bin-hadoop2.6.tgz: not in gzip format Do you want to terminate the 2 instances created by this operation?

nchammas commented 6 years ago

Starting with Spark 2.2.1, Spark is no longer hosted on S3. The project decided to move exclusively to the Apache mirror network. Try your launch with an earlier version of Spark, or install Flintrock from master.

For more information, see #238 and #249.