mattray / spiceweasel

Generates Chef knife commands from a simple JSON or YAML file.
Apache License 2.0
284 stars 65 forks source link

SSH Connection Timeout after 300 sec. #89

Closed mgdevstack closed 8 years ago

mgdevstack commented 8 years ago

Is there any configuration option to increase this connection timeout parameter ?

/root/.chefdk/gem/ruby/2.1.0/gems/mixlib-shellout-1.6.1/lib/mixlib/shellout/unix.rb:83:in `run_command': Command timed out after 300s: (Mixlib::ShellOut::CommandTimeout)
Command exceeded allowed execution time, process terminated

This is breaking server bootstrap process .. I tried with below configuraiton

"ec2 1":
              {
                     "run_list" : "role[xxx]",
                      "options" : "--session-timeout 10 -S key-pair -I ami-xxxxxx ...."
              }

But keeping this session timeout as 10 minute will cause failure if few package installation take more time than this and I want to put this in knife.rb configuraiton

In knife.rb configuraiton

knife[:ssh_attribute] = 'ConnectTimeout=900 StrictHostKeyChecking=no'

This is allowing me to wait for ssh connection to establish with aws instance. but breaking server creation and bootstrap it time exceeds 300 sec with top declared shell error.

Issue:

  1. How to tackle connection timeout issue.
  2. If I am launching 3 instance, only one server launch and other 2 fail as 300 sec already passed.
  3. How can I give nodename when I launch 3 instance with
{"ec2 3" :
    {
        run_list : ...
        "options" : " -N ServerCountNumber-21"
    } 
}

Nodes with naming ServerCountNumber-21 , ServerCountNumber-22, ServerCountNumber-23 should be launched.

mattray commented 8 years ago

It looks like mixlib-shellout has a default option for timing out of 600 seconds. I haven't looked through this code in a long time, but perhaps Spiceweasel needs to expose passing options to mixlib-shellout so we could change the timeout to something much larger.

https://github.com/chef/mixlib-shellout/blob/master/lib/mixlib/shellout.rb#L29

mgdevstack commented 8 years ago

Problem is resolved with {{ -T time_out_in_minute }} attribute in spiceweasel command. But this should be default to 0 ~infinite.. unless not explicitly mentioned, as most of cookbook may require 1+ hr bootstrap time (10-12 min to aws instance sshd, package installations, configurations, data syncing from other servers etc). chef exec spiceweasel -T 3600 -e path_to_infrastructure_json_file