TritonDataCenter / containerpilot

A service for autodiscovery and configuration of applications running in containers
Mozilla Public License 2.0
1.12k stars 136 forks source link

Question regarding 'when' directive #511

Closed sachinlala closed 6 years ago

sachinlala commented 6 years ago

Scenario

Service 'B' depends on service 'A' i.e. the startup for service B should not be initiated till service A is healthy. Both the services register in Consul.

Issue

We're using the 'when' directive to ensure the dependency check is done. What we're observing is:

Version

We're using the version 3.4.2 of containerpilot.

Configuration (app.json5)

{
  consul: "{{ .NOMAD_IP_cobalt }}:8500",
  jobs: [
    {
      name: "B",
      exec: "/var/tmp/startB.sh",
      port: 9092,
      when: {
        source: "A",
        each: "healthy"
      },
      health: {
        exec: "nc -v -z localhost 9092",
        interval: 10,
        ttl: 25
      },
      interfaces: [
          "eth0"
      ]
    }
  ]
}

Questions

  1. Is the above config the right way to replicate the preStart behaviour which was present in previous versions of containerpilot ?
  2. What is 'healthy' exactly mapped to ? I mean in Consul, the service status could be 'passing or 'failing' etc., but there is no 'healthy' status value there.
  3. What is the exact semantics of 'source' config (inside the 'when' directive) ?
    • we're currently providing simply the name of the service as it is registered in Consul e.g. service A is registered as name 'A' and give that as-is as a dependency (when -> source) in the app.json5 of service B
    • however, in the documentation it's advised (in 'name' section) that we should provide the name+hostname. If this is true, then it is a problem because hostname could change..
    • I'm specifically referring to these 2 points:

      Each instance of the service in Consul will have a unique ID made up from the name+hostname of the container. source is the source of the event that triggers the job.

For now, we have added a delay in the startup of service B to tactically circumvent the issue, but would like to utilize the 'when' directive rightfully.

jwreagor commented 6 years ago

Hello @sachinlala. It sounds like what you want is a watch and not an event trigger, have you looked at those?

Event triggers defined under jobs only interact with services under the control of ContainerPilot within a single container. A watch defines how ContainerPilot can watch for external service changes through Consul's service registry.

https://www.joyent.com/containerpilot/docs/configuration/watches

Since watches are a construct within Consul, the configuration defines them separately from jobs. You can still reference the watch from within a job's configuration, like so...

jobs: [
  {
    name: "B",
    exec: "/var/tmp/startB.sh",
    when: {
      source: "watch.A",
      each: "healthy"
    }
  }
],
watches: [
  {
    name: "A",
    interval: 3
  }
]

Notice the prepended hook under source, watch., this links the source of the event to an external watch in Consul.

Hope that answers your question or puts you on the right path.

sachinlala commented 6 years ago

Hi @cheapRoc - this does answer my question and it's worked good. Will be further helpful to include this semantic (specifically the point about watch.<service_name> in the source tag) in the main documentation.

Thanks a lot for your help.