Agent Spawner daemon

[praiskup]

Relates to #123

[siteshwar]

The process of starting a new osh worker is as follows:

• Request for a new machine.
• Wait for it to appear and get its hostname.
• Generate an osh worker configuration based on the hostname.
• Transfer the configuration to the new osh worker.
• Start osh worker daemon.

The last 3 steps should be done after the hostname is known and probably belong to a separate post_call_take() hook.

[praiskup]

The last 3 steps should be done after the hostname is known and probably belong to a separate post_call_take() hook.

We can have two hooks, but if they can be logically merged - the API would be simpler.

[praiskup]

@siteshwar what do you think about the current variant? It seems to be working with this configuration:

agent_groups:
  osh_workers:
    cmd_prepare: /bin/true
    cmd_terminate: echo noop
    cmd_converge_to: echo 5
    cmd_check_finished: exit $(( RANDOM % 15 ))
    cmd_try_release: exit $(( RANDOM % 5 ))
    tags:
      - kobo_worker

Of course you must be running resalloc server on a normal port, and have some pool defind with kobo_worker tag.