Install multiple agents on a node.

[tpbrown]

• node[:go][:agent][:instance_count] controls how many agents get installed on a node. Default is one per CPU.

This cookbook now replaces the init.d service and helper scripts that come in the Go debian package with versions from the cookbook. The cookbook versions are based on Go 13.2.0 and used as templates.

A single Go agent binary is installed, but multiple runtime directories are used.

/var/lib/go-agent[-n]

Tested on Ubuntu 12.04 LTS. /etc/defaults/go-agent-* likely needs updating for anything beyond Ubuntu. What's the proper way to find JAVA_HOME?!

[tpbrown]

-bump-

Anyone have time to review this pull request? I'd really rather not merge it myself ;-)

[scottmuc]

Thanks for the reminder. I'm going to have a chill evening tonight. Will look at it then.

[kPhilosopher]

Hey Tim,

I ran into an issue when I was running vagrant up with the following error:

[2013-07-16T21:01:42+00:00] FATAL: Chef::Exceptions::MultipleFailures: Multiple failures occurred:
* Chef::Exceptions::Exec occurred in chef run: service[go-agent-2] (go::agent line 113) had an error: Chef::Exceptions::Exec: /usr/sbin/update-rc.d go-agent-2 defaults returned 1, expected 0
* Mixlib::ShellOut::ShellCommandFailed occurred in delayed notification: service[go-agent-2] (go::agent line 113) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '255'
---- Begin output of /etc/init.d/go-agent-2 restart ----
STDOUT: Go is not running or you don't have permission to stop it
[Tue Jul 16 21:01:27 UTC 2013] using default settings from /etc/default/go-agent-2
Error starting Go Agent-2.
STDERR: /usr/share/go-agent/agent-2.sh: line 83: /var/lib/go-agent-2/go-agent-bootstrapper.log: Permission denied
/usr/share/go-agent/agent-2.sh: line 84: /var/lib/go-agent-2/go-agent-bootstrapper.log: Permission denied
/usr/share/go-agent/agent-2.sh: line 85: /var/lib/go-agent-2/go-agent-bootstrapper.log: Permission denied
/usr/share/go-agent/agent-2.sh: line 94: /var/lib/go-agent-2/go-agent-bootstrapper.log: Permission denied
---- End output of /etc/init.d/go-agent-2 restart ----
Ran /etc/init.d/go-agent-2 restart returned 255

So, I went into the VM and found this out:

root@vagrant:/home/vagrant# ls -asl /var/lib/go-agent-2/
total 12
4 drwxr-xr-x  3 root root 4096 Jul 16 21:01 .
4 drwxr-xr-x 37 root root 4096 Jul 16 21:01 ..
4 drwxr-xr-x  2 go   go   4096 Jul 16 21:01 config
root@vagrant:/home/vagrant# ls -asl /var/lib/go-agent
total 31596
    4 drwxr-xr-x  5 go   go       4096 Jul 16 21:02 .
    4 drwxr-xr-x 37 root root     4096 Jul 16 21:01 ..
 2392 -rw-rw-r--  1 go   go    2447101 Jul 16 21:01 87fae27b-88ee-457e-a6b2-88d3159ba93eagent-launcher.jar
    0 -rw-rw-r--  1 go   go          0 Jul 16 21:16 .agent-bootstrapper.running
26788 -rw-rw-r--  1 go   go   27427366 Jul 16 21:02 agent.jar
 2392 -rw-rw-r--  1 go   go    2447101 Jul 16 21:01 agent-launcher.jar
    4 drwxr-xr-x  2 go   go       4096 Jul 16 21:02 config
    4 drwxrwxr-x  3 go   go       4096 Jul 16 21:01 exploded_agent_launcher_dependencies
    4 -rw-r--r--  1 go   go        583 Jun 11 06:07 log4j.properties
    4 drwxrwxr-x  2 go   go       4096 Jul 16 21:02 pipelines

So, I put in a solution by adding a resource call to change the owner of /var/lib/go-agent-2/

 directory "/var/lib/go-agent#{suffix}" do
    mode '0755'
    owner 'go'
    group 'go'
    subscribes :create, "template[/etc/init.d/go-agent#{suffix}]"
    recursive true
    action :nothing
  end

Then, I ran into the following issue:

[2013-07-16T23:24:44+00:00] INFO: Processing service[go-agent-2] action enable (go::agent line 120)

================================================================================
Error executing action `enable` on resource 'service[go-agent-2]'
================================================================================

Chef::Exceptions::Exec
----------------------
/usr/sbin/update-rc.d go-agent-2 defaults returned 1, expected 0

Resource Declaration:

---------------------
# In /tmp/vagrant-chef-1/chef-solo-1/cookbooks/go/recipes/agent.rb

120:   service "go-agent#{suffix}" do
121:     supports :status => true, :restart => true, :reload => true, :start => true
122:     action [:enable, :nothing]
123:     subscribes :restart, "template[/etc/init.d/go-agent#{suffix}]"
124:     subscribes :restart, "template[/var/lib/go-agent#{suffix}/config/autoregister.properties]"
125:     subscribes :restart, "template[/etc/default/go-agent#{suffix}]"
126:   end
127:

Compiled Resource:

------------------

# Declared in /tmp/vagrant-chef-1/chef-solo-1/cookbooks/go/recipes/agent.rb:120:in `block in from_file'

service("go-agent-2") do
  action [:enable, :nothing]
  supports {:status=>true, :restart=>true, :reload=>true, :start=>true}
  retries 0
  retry_delay 2
  service_name "go-agent-2"
  pattern "go-agent-2"
  startup_type :automatic
  cookbook_name :go
  recipe_name "agent"
end

[2013-07-16T23:24:45+00:00] INFO: Running queued delayed notifications before re-raising exception

This was caused because the service enable call was happening before the init.d script was created. So, I had to add the :immediately part to the subscriptions.

[kPhilosopher]

Created pull request to your original code.

[tpbrown]

@kPhilosopher Appreciate the pull request!

I went through the problem you found myself, but I don't think the issue is delayed notifications entirely.

It breaks down into two things:

• directory "foo" doesn't set ownership recursively; adding the other directory statement appears to be the correct thing to do
• The service starting too soon isn't about the start, it's about the enable, Changing when we enable the service fixes the problem, and doesn't require setting everything to immediate notification.

I'm going to close the pull request and commit what i sorted out. Please let me know if you see any other problems.

Cheers!