search

amplab / spark-ec2

Scripts used to setup a Spark cluster on EC2
Apache License 2.0
393 stars 299 forks source link

ap-northeast-2 seoul region support #94

Closed statkclee closed 7 years ago

statkclee commented 7 years ago

I've got the following error such that I've looked at the code and found out no seoul region. When are you going to support seoul region?

vagrant@sparkvm:~/spark-ec2$ ./spark-ec2 --key-pair=victor-kr --identity-file=/ etc/victor-kr.pem --instance-type=m4.large -s 1 --region=ap-northeast-2 --zone= ap-northeast-2c launch spark-cluster-mu-legend Setting up security groups... Traceback (most recent call last): File "./spark_ec2.py", line 1534, in main() File "./spark_ec2.py", line 1526, in main real_main() File "./spark_ec2.py", line 1355, in real_main (master_nodes, slave_nodes) = launch_cluster(conn, opts, cluster_name) File "./spark_ec2.py", line 500, in launch_cluster master_group = get_or_make_group(conn, cluster_name + "-master", opts.vpc_id ) File "./spark_ec2.py", line 361, in get_or_make_group groups = conn.get_all_security_groups() AttributeError: 'NoneType' object has no attribute 'get_all_security_groups'

nchammas commented 7 years ago

Related: #13, #78, #80

spark-ec2 depends on custom AMIs which are rarely updated. I doubt support for the Seoul region will be added anytime soon, but I would love to be proven wrong about that.

If you are interested in trying a similar tool to spark-ec2 that just depends on the default Amazon Linux AMI, I recommend my project, Flintrock. It should work with Amazon Linux in any region out of the box.

shivaram commented 7 years ago

Unfortunately given the other commitments I have I am not sure when the AMIs will be updated. Meanwhile I wholeheartedly recommend checking out Flintrock for newer regions.

statkclee commented 7 years ago

I see... Thank you for recommendations.

nhlumen commented 7 years ago

Could you please include eu-west-2 to your supported regions? Thanks!