search

amplab / spark-ec2

Scripts used to setup a Spark cluster on EC2
Apache License 2.0
392 stars 299 forks source link

tachyon not supported? #113

Open ddsky opened 6 years ago

ddsky commented 6 years ago

I started instances successfully with the ./spark-ec2 script but at the end I don't get the information like http://master-hostname:8080 - I assume because I get No valid Tachyon version found; Tachyon won't be set up

I'm running this on windows, is this not supported?

Is this tachyon necessary, I see on AWS that the instances were started successfully. Is there another way I can get into the web ui?

guidovansteen commented 6 years ago

For some reason the tachyon archive is missing at the url that the script uses.

I guess tachyon is not really necessary. I edited the "./spark_ec2.py" file, and applied these changes:

Old code: modules = ['spark', 'ephemeral-hdfs', 'persistent-hdfs', 'mapreduce', 'spark-standalone', 'tachyon', 'rstudio']

New code: modules = ['spark', 'ephemeral-hdfs', 'persistent-hdfs', 'mapreduce', 'spark-standalone']

After this I encountered no more errors, and I could connect to http://master-hostname:8080.

However, this spark-ec2 project seems to be pretty abandoned. You may be better off trying flintrock.