Can create multiple apps with the same service port

ejether commented 8 years ago

We are running: Mesos @ 0.25.0-0.2.70.ubuntu1404 Marathon @ 0.14.0-1.0.450.ubuntu1404 Marathon-lb in docker @ tag:latest as of 1.28.2016 on ubuntu 14.04 in GCE

We are currently able to create multiple apps with the same service port both via the api and the ui If I understand this bug (https://github.com/mesosphere/marathon/issues/1002) that should not be the case.

xianlubird commented 8 years ago

Does the hostport or ports are random assigned?

ejether commented 8 years ago

I'm sorry, I'm not sure what you are asking so I'll try to explain a little bit more. We have a service running in a docker container with the following port configuration:

{ "id": "/our-service", "cpus": 1, "cmd": "run service $PORT", "mem": 512, "requirePorts": false, "instances": 1, "ports": [9093], "constraints": [["hostname", "UNIQUE"]], "healthChecks": [ { "protocol": "HTTP", "path": "/ping", "portIndex": 0, "gracePeriodSeconds": 20, "intervalSeconds": 5, "timeoutSeconds": 2, "maxConsecutiveFailures": 3 } ], }

we changed the app id and started an app with this configuration but did not stop the first app: { "id": "/our/service", "cpus": 1, "cmd": "run service $PORT", "mem": 512, "requirePorts": false, "instances": 1, "ports": [9093], "constraints": [["hostname", "UNIQUE"]], "healthChecks": [ { "protocol": "HTTP", "path": "/ping", "portIndex": 0, "gracePeriodSeconds": 20, "intervalSeconds": 5, "timeoutSeconds": 2, "maxConsecutiveFailures": 3 } ], }

We expected it to fail because the port 9093 is the same in both services. Marathon started two services, each on their own host port, say 31079 and 31775 respectively, and we were able to reach the different versions of the service on the host ports. However, the haproxy configuration then had two different front end configurations listening on port 9093 and two different backends configured, one for each app id.

To test this behavior, while the two apps running on 9093 I also created several apps in the ui and assigned the port value to 9093 for each one of them.

According to the bug, and previously seen behavior, Marathon should not allow two apps to have the same service port.

Poltergeist commented 8 years ago

I checked this Issue, If I create an app with a serviceport which is taken then the service port will be automatically be reassigned to another free port. Could you please reensure that the ports on both applications are the same?

ejether commented 8 years ago

Perhaps I am confused about service port vs host port.
Based on my understanding:

Service port is what HAProxy will bind to.
Host port is the auto assigned 3xxxx port

Based on the above statements: I am not seeing it auto assign a Service Port I am seeing it auto assign a Host Port

Three apps running python simple web server and a unique file per app

{ "apps": [ { "acceptedResourceRoles": null, "args": null, "backoffFactor": 1.15, "backoffSeconds": 1, "cmd": "echo 'test2' > test2; python -m SimpleHTTPServer $PORT", "constraints": [], "container": null, "cpus": 0.1, "dependencies": [], "deployments": [], "disk": 0, "env": {}, "executor": "", "healthChecks": [], "id": "/test2", "instances": 1, "ipAddress": null, "labels": {}, "maxLaunchDelaySeconds": 3600, "mem": 16, "ports": [ 9093 ], "requirePorts": false, "storeUrls": [], "tasksHealthy": 0, "tasksRunning": 1, "tasksStaged": 0, "tasksUnhealthy": 0, "upgradeStrategy": { "maximumOverCapacity": 1, "minimumHealthCapacity": 1 }, "uris": [], "user": null, "version": "2016-01-29T16:19:45.626Z", "versionInfo": { "lastConfigChangeAt": "2016-01-29T16:19:45.626Z", "lastScalingAt": "2016-01-29T16:19:45.626Z" } }, { "acceptedResourceRoles": null, "args": null, "backoffFactor": 1.15, "backoffSeconds": 1, "cmd": "echo 'test3' > test3; python -m SimpleHTTPServer $PORT", "constraints": [], "container": null, "cpus": 0.1, "dependencies": [], "deployments": [], "disk": 0, "env": {}, "executor": "", "healthChecks": [], "id": "/test3", "instances": 1, "ipAddress": null, "labels": {}, "maxLaunchDelaySeconds": 3600, "mem": 16, "ports": [ 9093 ], "requirePorts": false, "storeUrls": [], "tasksHealthy": 0, "tasksRunning": 1, "tasksStaged": 0, "tasksUnhealthy": 0, "upgradeStrategy": { "maximumOverCapacity": 1, "minimumHealthCapacity": 1 }, "uris": [], "user": null, "version": "2016-01-29T16:19:58.208Z", "versionInfo": { "lastConfigChangeAt": "2016-01-29T16:19:58.208Z", "lastScalingAt": "2016-01-29T16:19:58.208Z" } }, { "acceptedResourceRoles": null, "args": null, "backoffFactor": 1.15, "backoffSeconds": 1, "cmd": "echo 'test1' > test1; python -m SimpleHTTPServer $PORT", "constraints": [], "container": null, "cpus": 0.1, "dependencies": [], "deployments": [], "disk": 0, "env": {}, "executor": "", "healthChecks": [], "id": "/test1", "instances": 1, "ipAddress": null, "labels": {}, "maxLaunchDelaySeconds": 3600, "mem": 16, "ports": [ 9093 ], "requirePorts": false, "storeUrls": [], "tasksHealthy": 0, "tasksRunning": 1, "tasksStaged": 0, "tasksUnhealthy": 0, "upgradeStrategy": { "maximumOverCapacity": 1, "minimumHealthCapacity": 1 }, "uris": [], "user": null, "version": "2016-01-29T16:20:50.642Z", "versionInfo": { "lastConfigChangeAt": "2016-01-29T16:20:50.642Z", "lastScalingAt": "2016-01-29T16:20:50.642Z" } } ] }

Resulted in this HAProxy configuration in Marathon-lb:

listen stats
  bind 0.0.0.0:1936
  mode http
  stats enable
  stats uri /
  monitor-uri /_haproxy_health_check
frontend marathon_http_appid_in
  bind *:9091
  mode http

frontend test1_9093
  bind *:9093
  mode tcp
  use_backend test1_9093

frontend test2_9093
  bind *:9093
  mode tcp
  use_backend test2_9093

frontend test3_9093
  bind *:9093
  mode tcp
  use_backend test3_9093

backend test1_9093
  balance roundrobin
  mode tcp
  server mesos-slave-2_31666 192.160.33.13:31666

backend test2_9093
  balance roundrobin
  mode tcp
  server mesos-slave-1_31989 192.160.33.12:31989

backend test3_9093
  balance roundrobin
  mode tcp
  server mesos-slave-1_31675 192.160.33.12:31675

When I visit apps test1,test2,test3 at 192.160.33.13:31666,192.160.33.12:31989,192.160.33.12:31675 respectively, I see the unique files where they should be.
When I visit 192.160.33.12:9093, I randomly get test1,test2,test3 (Which makes sense based on the HAProxy config)
I reverted to 0.13.0-1.0.440.ubuntu1404 and was able to create a test4 app with the same configuration
I reverted to 0.11.1-1.0.432.ubuntu1404 and was able to create a test5 app with the same configuration
I reverted to 0.10.0-1.0.409.ubuntu1404 and was able to create a test5 app with the same configuration
On a suggestion for a co-worker I renamed an app (0.13.0-1.0.440.ubuntu1404) which he indicated is one way he had seen the port conflict error before. I created a new app with the new name and left the original app in place. Both still have the same service port.

Based on the bug I referenced, I shouldn't be able to create multiple apps with the same service port. I can see logic in having it both ways since if you could use the same service port, it would be very easy to red/green deployments. Can you tell me if what I am seeing is expected behavior?

gkleiman commented 8 years ago

@ejether

According to the documentation, what you describe is the desired behaviour:

Service Ports Assignment When you create a new application in Marathon (either through the REST API or the front end), you may assign one or more service ports to it. You can specify all valid port numbers as service ports or you can use 0 to indicate that Marathon should allocate free service ports to the app automatically. If you do choose your own service port, you have to ensure yourself that it is unique across all of your applications.

ejether commented 8 years ago

@gkleiman thanks for confirming that. It might be worth looking at https://github.com/mesosphere/marathon/issues/1002 and talking to @ConnorDoyle if you can. Based on my read, there was some code merged https://github.com/mesosphere/marathon/pull/1163 to prevent that behavior.

Thanks again

d2iq-archive / marathon

Can create multiple apps with the same service port #3122