search

dask / dask-ec2

Start a cluster in EC2 for dask.distributed
106 stars 37 forks source link

Can't start a cluster from an AMI #84

Open arokem opened 7 years ago

arokem commented 7 years ago

I created the AMI by starting a dask-ec2 cluster, logging into the 0th node, installing other software dependencies, and then creating an AMI from this node.

Then, when starting a dask-ec2 cluster pointing to this AMI, I get the following:

D-173-250-173-46:~  $dask-ec2 up --keyname afq-east-1 --keypair ~/afq-east-1.pem --type r4.xlarge --name dask-ivim --tags foo:bar --ami ami-fb9111ed
A file named cluster.yaml already exists, proceeding will overwrite this file. Continue? [y/N]: y
DEBUG: Searching for default VPC
DEBUG: Default VPC found - Using VPC ID: vpc-0a633d6d
DEBUG: Searching for default subnet in VPC vpc-0a633d6d
DEBUG: Default subnet found - Using Subnet ID: subnet-bf26b7da
Launching nodes
DEBUG: Checking that keyname 'afq-east-1' exists on EC2
DEBUG: Checking that security group 'dask-ec2-default' exists on EC2
DEBUG: Getting all security groups and filtering by VPC ID vpc-0a633d6d and name dask-ec2-default
DEBUG: Found Security groups: [ec2.SecurityGroup(id='sg-97072dea')]
DEBUG: Getting all security groups and filtering by VPC ID vpc-0a633d6d and name dask-ec2-default
DEBUG: Found Security groups: [ec2.SecurityGroup(id='sg-97072dea')]
DEBUG: Creating 4 instances on EC2
DEBUG: Tagging instance 'i-0dfc9e4469ebd8be3'
DEBUG: Tagging instance 'i-03b1a55e43f1c82e2'
DEBUG: Tagging instance 'i-0ad8a5e88f72e458c'
DEBUG: Tagging instance 'i-0d3ee89434503b0b6'
Checking SSH connection to nodes
DEBUG: Checking ssh connection for 34.205.28.59
DEBUG: Attempt 1/10 of function 'check_ssh' failed
DEBUG: Checking ssh connection for 34.205.28.59
DEBUG: Attempt 2/10 of function 'check_ssh' failed
DEBUG: Checking ssh connection for 34.205.28.59
DEBUG: Attempt 3/10 of function 'check_ssh' failed
DEBUG: Checking ssh connection for 34.205.28.59
DEBUG: Attempt 4/10 of function 'check_ssh' failed
DEBUG: Checking ssh connection for 34.205.28.59
DEBUG: Running command bash -c 'ls' on '34.205.28.59'
DEBUG: Checking ssh connection for 54.86.139.96
DEBUG: Running command bash -c 'ls' on '54.86.139.96'
DEBUG: Checking ssh connection for 34.200.214.76
DEBUG: Running command bash -c 'ls' on '34.200.214.76'
DEBUG: Checking ssh connection for 54.91.36.174
DEBUG: Running command bash -c 'ls' on '54.91.36.174'
+------------------+-----------+
| Node IP          | SSH check |
+==================+===========+
| 34.200.214.76:22 | True      |
| 54.91.36.174:22  | True      |
| 34.205.28.59:22  | True      |
| 54.86.139.96:22  | True      |
+------------------+-----------+
Bootstrapping salt master
DEBUG: Running command sudo -S bash -c 'curl -sS -L https://bootstrap.saltstack.com | sh -s -- -d -X -M -N stable' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'curl -L https://bootstrap.saltstack.com | sh -s -- -d -X -M -N -P -L -p salt-api stable' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/templates/auto_accept.conf to /tmp/.__tmp_copy
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy /etc/salt/master.d/auto_accept.conf' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'apt-get install -y python-pip libssl-dev libffi-dev python-dev' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'pip install --upgrade pip packaging appdirs six' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'pip install cherrypy' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'pip install PyOpenSSL==16.2.0' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'salt-call --local tls.create_self_signed_cert' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/templates/rest_cherrypy.conf to /tmp/.__tmp_copy
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy /etc/salt/master.d/rest_cherrypy.conf' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/templates/external_auth.conf to /tmp/.__tmp_copy
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy /etc/salt/master.d/external_auth.conf' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'id -u saltdev &>/dev/null || useradd -p $(openssl passwd -1 saltdev) saltdev' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'service salt-master restart' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'service salt-api restart' on '34.205.28.59'
Bootstrapping salt minions
DEBUG: Installing salt-minion on all the nodes
DEBUG: Running command sudo -S bash -c 'curl -L https://bootstrap.saltstack.com | sh -s -- -d -X -P -L -A 34.205.28.59 -i node-2 stable' on '34.200.214.76'
DEBUG: Running command sudo -S bash -c 'curl -L https://bootstrap.saltstack.com | sh -s -- -d -X -P -L -A 34.205.28.59 -i node-3 stable' on '54.91.36.174'
DEBUG: Running command sudo -S bash -c 'curl -L https://bootstrap.saltstack.com | sh -s -- -d -X -P -L -A 34.205.28.59 -i node-0 stable' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'curl -L https://bootstrap.saltstack.com | sh -s -- -d -X -P -L -A 34.205.28.59 -i node-1 stable' on '54.86.139.96'
DEBUG: Configuring salt-mine on the salt minions
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/templates/mine_functions.conf to /tmp/.__tmp_copy
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/templates/mine_functions.conf to /tmp/.__tmp_copy
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/templates/mine_functions.conf to /tmp/.__tmp_copy
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/templates/mine_functions.conf to /tmp/.__tmp_copy
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy /etc/salt/minion.d/mine.conf' on '34.200.214.76'
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy /etc/salt/minion.d/mine.conf' on '54.86.139.96'
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy /etc/salt/minion.d/mine.conf' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy /etc/salt/minion.d/mine.conf' on '54.91.36.174'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '34.200.214.76'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '54.86.139.96'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '54.91.36.174'
DEBUG: Restarting the salt-minion service
DEBUG: Running command sudo -S bash -c 'service salt-minion restart' on '34.200.214.76'
DEBUG: Running command sudo -S bash -c 'service salt-minion restart' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'service salt-minion restart' on '54.91.36.174'
DEBUG: Running command sudo -S bash -c 'service salt-minion restart' on '54.86.139.96'
Uploading salt formulas
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt to /srv/salt
DEBUG: Running command sudo -S bash -c 'mkdir -p /srv/salt' on '34.205.28.59'
DEBUG: Creating directory /tmp/.__tmp_copy mode=511
DEBUG: Creating directory /tmp/.__tmp_copy/conda mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/conda to /tmp/.__tmp_copy/conda
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/conda' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/conda/debug.sls to /tmp/.__tmp_copy/conda/debug.sls
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/conda/init.sls to /tmp/.__tmp_copy/conda/init.sls
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/conda/settings.sls to /tmp/.__tmp_copy/conda/settings.sls
DEBUG: Creating directory /tmp/.__tmp_copy/conda/templates mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/conda/templates to /tmp/.__tmp_copy/conda/templates
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/conda/templates' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/conda/templates/conda.sh to /tmp/.__tmp_copy/conda/templates/conda.sh
DEBUG: Creating directory /tmp/.__tmp_copy/dask mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask to /tmp/.__tmp_copy/dask
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/dask' on '34.205.28.59'
DEBUG: Creating directory /tmp/.__tmp_copy/dask/distributed mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask/distributed to /tmp/.__tmp_copy/dask/distributed
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/dask/distributed' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask/distributed/debug.sls to /tmp/.__tmp_copy/dask/distributed/debug.sls
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask/distributed/init.sls to /tmp/.__tmp_copy/dask/distributed/init.sls
DEBUG: Creating directory /tmp/.__tmp_copy/dask/distributed/scheduler mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask/distributed/scheduler to /tmp/.__tmp_copy/dask/distributed/scheduler
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/dask/distributed/scheduler' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask/distributed/scheduler/init.sls to /tmp/.__tmp_copy/dask/distributed/scheduler/init.sls
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask/distributed/settings.sls to /tmp/.__tmp_copy/dask/distributed/settings.sls
DEBUG: Creating directory /tmp/.__tmp_copy/dask/distributed/templates mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask/distributed/templates to /tmp/.__tmp_copy/dask/distributed/templates
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/dask/distributed/templates' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask/distributed/templates/dask-scheduler.conf to /tmp/.__tmp_copy/dask/distributed/templates/dask-scheduler.conf
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask/distributed/templates/dask-worker.conf to /tmp/.__tmp_copy/dask/distributed/templates/dask-worker.conf
DEBUG: Creating directory /tmp/.__tmp_copy/dask/distributed/worker mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask/distributed/worker to /tmp/.__tmp_copy/dask/distributed/worker
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/dask/distributed/worker' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/dask/distributed/worker/init.sls to /tmp/.__tmp_copy/dask/distributed/worker/init.sls
DEBUG: Creating directory /tmp/.__tmp_copy/java mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/java to /tmp/.__tmp_copy/java
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/java' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/java/debug.sls to /tmp/.__tmp_copy/java/debug.sls
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/java/init.sls to /tmp/.__tmp_copy/java/init.sls
DEBUG: Creating directory /tmp/.__tmp_copy/java/openjdk mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/java/openjdk to /tmp/.__tmp_copy/java/openjdk
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/java/openjdk' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/java/openjdk/env.sls to /tmp/.__tmp_copy/java/openjdk/env.sls
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/java/openjdk/init.sls to /tmp/.__tmp_copy/java/openjdk/init.sls
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/java/openjdk/settings.sls to /tmp/.__tmp_copy/java/openjdk/settings.sls
DEBUG: Creating directory /tmp/.__tmp_copy/java/openjdk/templates mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/java/openjdk/templates to /tmp/.__tmp_copy/java/openjdk/templates
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/java/openjdk/templates' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/java/openjdk/templates/java.sh to /tmp/.__tmp_copy/java/openjdk/templates/java.sh
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/java/settings.sls to /tmp/.__tmp_copy/java/settings.sls
DEBUG: Creating directory /tmp/.__tmp_copy/jupyter mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/jupyter to /tmp/.__tmp_copy/jupyter
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/jupyter' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/jupyter/init.sls to /tmp/.__tmp_copy/jupyter/init.sls
DEBUG: Creating directory /tmp/.__tmp_copy/jupyter/notebook mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/jupyter/notebook to /tmp/.__tmp_copy/jupyter/notebook
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/jupyter/notebook' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/jupyter/notebook/init.sls to /tmp/.__tmp_copy/jupyter/notebook/init.sls
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/jupyter/notebook/settings.sls to /tmp/.__tmp_copy/jupyter/notebook/settings.sls
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/jupyter/settings.sls to /tmp/.__tmp_copy/jupyter/settings.sls
DEBUG: Creating directory /tmp/.__tmp_copy/jupyter/templates mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/jupyter/templates to /tmp/.__tmp_copy/jupyter/templates
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/jupyter/templates' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/jupyter/templates/jupyter-notebook.conf to /tmp/.__tmp_copy/jupyter/templates/jupyter-notebook.conf
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/jupyter/templates/jupyter_notebook_config_py to /tmp/.__tmp_copy/jupyter/templates/jupyter_notebook_config_py
DEBUG: Creating directory /tmp/.__tmp_copy/supervisor mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/supervisor to /tmp/.__tmp_copy/supervisor
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/supervisor' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/supervisor/init.sls to /tmp/.__tmp_copy/supervisor/init.sls
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/supervisor/settings.sls to /tmp/.__tmp_copy/supervisor/settings.sls
DEBUG: Creating directory /tmp/.__tmp_copy/system mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/system to /tmp/.__tmp_copy/system
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/system' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/system/base.sls to /tmp/.__tmp_copy/system/base.sls
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/system/macros.sls to /tmp/.__tmp_copy/system/macros.sls
DEBUG: Creating directory /tmp/.__tmp_copy/system/templates mode=511
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/system/templates to /tmp/.__tmp_copy/system/templates
DEBUG: Running command sudo -S bash -c 'mkdir -p /tmp/.__tmp_copy/system/templates' on '34.205.28.59'
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/system/templates/limits.conf to /tmp/.__tmp_copy/system/templates/limits.conf
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/salt/top.sls to /tmp/.__tmp_copy/top.sls
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy/* /srv/salt' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '34.205.28.59'
DEBUG: Uploading directory /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/pillar to /srv/pillar
DEBUG: Running command sudo -S bash -c 'mkdir -p /srv/pillar' on '34.205.28.59'
DEBUG: Creating directory /tmp/.__tmp_copy mode=511
DEBUG: Uploading file /Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/formulas/pillar/top.sls to /tmp/.__tmp_copy/top.sls
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy/* /srv/pillar' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '34.205.28.59'
Uploading conda and cluster settings
DEBUG: Uploading file /var/folders/jz/4k438hl91v7gvdd592ct4lpc0000gn/T/tmpj5bqiikn to /tmp/.__tmp_copy
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy /srv/pillar/conda.sls' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '34.205.28.59'
DEBUG: Uploading file /var/folders/jz/4k438hl91v7gvdd592ct4lpc0000gn/T/tmp53l35go2 to /tmp/.__tmp_copy
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy /srv/pillar/cluster.sls' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '34.205.28.59'
+---------+----------------------+-----------------+
| Node ID | # Successful actions | # Failed action |
+=========+======================+=================+
| node-0  | 6                    | 0               |
+---------+----------------------+-----------------+
DEBUG: Uploading file /var/folders/jz/4k438hl91v7gvdd592ct4lpc0000gn/T/tmpa5e7v51x to /tmp/.__tmp_copy
DEBUG: Running command sudo -S bash -c 'cp -rf /tmp/.__tmp_copy /srv/pillar/dask.sls' on '34.205.28.59'
DEBUG: Running command sudo -S bash -c 'rm -rf /tmp/.__tmp_copy' on '34.205.28.59'
Installing scheduler
Traceback (most recent call last):
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/cli/main.py", line 23, in start
    cli(obj={})
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/cli/main.py", line 197, in up
    dask=dask, notebook=notebook, nprocs=nprocs, source=source)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/cli/main.py", line 343, in provision
    ctx.invoke(dask_install, filepath=filepath, nprocs=nprocs, source=source)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/cli/daskd.py", line 74, in dask_install
    response = print_state(output)
  File "/Users/arokem/anaconda3/lib/python3.5/site-packages/dask_ec2/cli/main.py", line 371, in print_state
    data.extend(response.aggregated_to_table(agg=len))
AttributeError: 'NoneType' object has no attribute 'aggregated_to_table'