search

dask / dask-ec2

Start a cluster in EC2 for dask.distributed
106 stars 37 forks source link

Rocky start with dask-ec2 - neither default ami, nor Ubuntu 14.04, 16.04, 17.04 work #99

Open sotte opened 7 years ago

sotte commented 7 years ago

Dask-ec2 looks like a gread project! Currently the initial user experience is pretty rocky unfortunately. Neither the default ami, nor amis for Ubuntu 14.04, 16.04, and 17.04 allow me to setup a cluster.

I'll run you through the steps I tried. I used the current version from master 1849efc68545a0ade81b1787da75dcb4df913037 for the experiments.

Default AMI

The default ami ami-d05e75b8 is for us-east-1 and does not work for Europe. Ok, no problem, I'll just use an appropriate ami for Europe (see further down).

Maybe dask-ec2 should suggest a resonable ami based on the region?

Ubuntu 14.04 for Europe

dask-ec2 up \
  --region-name eu-central-1 \
  --keyname some_key_name \
  --tags tag:test \
  --keypair some_key_pair \
  --name some_name \
  --type m4.large \
  --ami ami-ec832a83 \
  --count 4

This runs for a while, creates 4 instances, gives me access to the dask status frontend and the jupyter notebook (via http), but the scheduler does not know about the workers.

screen shot 2017-09-01 at 16 02 25

Ubuntu 16.04

Doing the same with newer ami: ami-958128fa (Ubuntu 16.04, eu-central-1)

dask-ec2 up \
  --region-name eu-central-1 \
  --keyname some_key_name \
  --tags tag:test \
  --keypair some_key_pair \
  --name some_name \
  --type m4.large \
  --ami ami-958128fa \
  --count 4

This leads to the ssl error #98 

...
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2171)

Ubuntu 17.04

Like 16.04, this leads to

...
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2171)
mrocklin commented 7 years ago

Hi @sotte , sorry that you've had a poor experience here. Unfortunately the primary developer of this repository, @danielfrg, is mostly occupied with other things. He might stop by and fix things like this, but we probably can't depend on it. Are you able / interested in helping to resolve some of the issues that you've raised here? If so, that would be greatly appreciated.

sotte commented 7 years ago

Hey @mrocklin, thanks for the quick reply. I'm not complaining about dask-ec2 :) It looks like a great project. I was just trying to summarize the problem I had when checking out the project hoping that it would help to fix some of the issues.

Sadly I don't have the time to help out nor do I have any salt experience.

jamesbehzadi commented 7 years ago

Hi @mrocklin @danielfrg

I'm a big fan of Dask. Works seamlessly on my local machine. But when I try to use dask-ec2 in Australia (ap-southeast-2), I pretty much ran into the same issues as @sotte.

I'd appreciate it, if you could please provide some help. Thanks!

Some details:

Bootstraping salt master DEBUG: Running command sudo -S bash -c 'curl -sS -L https://bootstrap.saltstack.com | sh -s -- -d -X -M -N stable' on '52.62.9.214' DEBUG: Attempt 1/3 of function '__install_salt_master' failed DEBUG: Running command sudo -S bash -c 'curl -sS -L https://bootstrap.saltstack.com | sh -s -- -d -X -M -N stable' on '52.62.9.214' DEBUG: Attempt 2/3 of function '__install_salt_master' failed DEBUG: Running command sudo -S bash -c 'curl -sS -L https://bootstrap.saltstack.com | sh -s -- -d -X -M -N stable' on '52.62.9.214' DEBUG: Attempt 3/3 of function '__install_salt_master' failed ERROR: 'str' object has no attribute 'decode' Couldn't bootstrap salt-master. Error is above (maybe try again)

DEBUG: Running command sudo -S bash -c 'pip install cherrypy' on '13.210.125.77' DEBUG: Attempt 1/3 of function '__install_salt_rest_api' failed DEBUG: Running command sudo -S bash -c 'pip install cherrypy' on '13.210.125.77' DEBUG: Attempt 2/3 of function '__install_salt_rest_api' failed DEBUG: Running command sudo -S bash -c 'pip install cherrypy' on '13.210.125.77' DEBUG: Attempt 3/3 of function '__install_salt_rest_api' failed ERROR: 'str' object has no attribute 'decode' Couldn't install CherryPy. Error is above (maybe try again)