amplab / spark-ec2

Scripts used to setup a Spark cluster on EC2
Apache License 2.0
393 stars 299 forks source link

Support for S3 V4 #102

Closed bfleming-ciena closed 7 years ago

bfleming-ciena commented 7 years ago

I realize this project is not being maintained any longer, but if anyone can comment to help me avoid a lot of work short term, I'm trying to figure out how to replace the aws S3 driver jets3t with a newer version to support the V4 S3 signature that is only available on new regions like frankfurt and canada.

I believe if I can replace the jetst 0.9.0 with 0.9.4 it may work. I attempted to use spark-submit --jars and pass .0.9.4 to no avail.

So, as a charity case, if anyone has some clues I would appreciate it greatly.

bfleming-ciena commented 7 years ago

K

ty