search

amplab / spark-ec2

Scripts used to setup a Spark cluster on EC2
Apache License 2.0
393 stars 299 forks source link

Hadoop 2.6 support for Spark 1.6 #82

Open anfeng opened 7 years ago

anfeng commented 7 years ago

Is there any reason that you only support Hadoop 2.4 for Spark 1.6? It will be lovely if you could also support Hadoop 2.6.

WIgor commented 7 years ago

Actually, I've made some dirty hacks here https://github.com/grammarly/spark-ec2 for my own use. Not so complex and it works, in my case. I needed in s3a support that was included to hadoop 2.6.

shivaram commented 7 years ago

@wigor would you be interested in sending a PR for this ?

WIgor commented 7 years ago

@shivaram Yes, I thought to update this to latest spark version. And then will send PR.

anfeng commented 7 years ago

propose a PR #88