how are multi-port jobs handled?

[brentley]

https://github.com/CiscoCloud/mesos-consul/blob/master/mesos/mesos.go#L121

Looking at this, I am not clear how a job/service that has multiple ports will be registered. How will they be differentiated when it comes to service registration in consul?

Thanks for the work so far!

[ChrisAubuchon]

They are currently registered with the same service name but with a different service ID. The service ID is built as "::"

Example:

Test service registration files:

test1.json:
{
  "ID": "test1",
  "Name": "test",
  "Address": "127.0.0.1",
  "Port": 80
}
test2.json
{
  "ID": "test2",
  "Name": "test",
  "Address": "127.0.0.1",
  "Port": 443
}

Registered via:

# curl -X PUT -d @test1.json localhost:8500/v1/agent/service/register
# curl -X PUT -d @test2.json localhost:8500/v1/agent/service/register

Some dig'ing:

# dig +noall +answer @127.0.0.1 -p 8600 test.service.consul ANY
test.service.consul.    0   IN  A   127.0.0.1
# dig +noall +answer @127.0.0.1 -p 8600 test.service.consul SRV
test.service.consul.    0   IN  SRV 1 1 80 default.node.vagrant.consul.
test.service.consul.    0   IN  SRV 1 1 443 default.node.vagrant.consul.

[memelet]

This doesn't seem a good fit for when the ports are dynamically allocated. There would be no way for the client of the service to distinguish between the services. It would be nice to have a mechanism like like registrator, where environment variables (or some kind metadata that can be defined in mesos) are used to name the services.

[ChrisAubuchon]

My intention was/is to use Mesos task labels as the "registrator environment variable" replacement. Unfortunately there doesn't seem to be a way to get port mappings out of Mesos. To use an example app from the CiscoCloud/microservices-infratstructure project:

{
  "container": {
    "type": "DOCKER",
    "docker": {
      "network": "BRIDGE",
      "image": "keithchambers/docker-hello-world",
      "portMappings": [
        {
          "containerPort": 80,
          "protocol": "tcp"
        }
      ]
    }
  },
  "id": "hello-world",
  "instances": 1,
  "cpus": 0.1,
  "mem": 128
}

This is the information that we get from Mesos:

{
  "executor_id": "",
  "framework_id": "20150615-175443-16777343-5050-629-0000",
  "id": "hello-world.6d13f5eb-1389-11e5-a693-f63bd36ba682",
  "labels": [],
  "name": "hello-world",
  "resources": {
    "cpus": 0.1,
    "disk": 0,
    "mem": 128,
    "ports": "[4001-4001]"
  },
  "slave_id": "20150615-180504-251789322-15050-8375-S0",
  "state": "TASK_RUNNING",
  "statuses": [
    {
      "state": "TASK_RUNNING",
      "timestamp": 1434391819.31373
    }
  ]
}

All we get is the final port. Nothing to map back to the containerPort which is what we need.

[memelet]

And mesos returns two ports when the container only declared one.

[stevendborrelli]

As an FYI. registrator has just implemented docker label support: https://github.com/gliderlabs/registrator/pull/192

This is definitely an area where Mesos needs to export more information from the containers.

[ChrisAubuchon]

It only returns the one port. The ports entry is a range. It says that the task is using all of the ports from 4001 to 4001. So just 4001.

[stevendborrelli]

@brentley @memelet FYI, in https://github.com/CiscoCloud/mesos-consul/pull/35, mesos-consul added support for DiscoveryInfo.

[a-nldisr]

We can confirm that DiscoveryInfo allows registration for multiple ports. We use configured Aurora to allow this, its working for us.