search

signalfx / maestro-ng

Orchestration of Docker-based, multi-host environments
https://signalfx.com
Apache License 2.0
683 stars 83 forks source link

Lifecycle checks do not work when ports are bound to a specific interface or IP address on the host #217

Closed petrkalina closed 4 years ago

petrkalina commented 4 years ago 
ships:
    db:
        # address to which the containers' exposed ports bind
        ip: 10.66.22.1
        docker_port: 2376
        # address to which the docker daemon binds
        endpoint: 10.76.23.193
        tls: true
        tls_verify: true
        tls_ca_cert: /opt/j4care/conf/maestro/ship-tls/ca.crt
        tls_key: /opt/j4care/conf/maestro/ship-tls/ship.key
        tls_cert: /opt/j4care/conf/maestro/ship-tls/ship.crt
        ssl_version: PROTOCOL_TLSv1_2

services:

    db-psql:
        image: postgres
        instances:
            db-psql1:
                ship: db
                ports:
                    postgresql:
                        exposed: 5432
                        external: [ 10.66.22.1, 5432 ]
                lifecycle:
                    running: [{type: tcp, port: postgresql}]

.. lifecycle does not detect the port being open and this command hangs

j4care@jms1dvlvnadb01:/opt/j4care/maestro$ maestro -f db-psql.yaml restart db-psql1
  #  INSTANCE                               SERVICE              SHIP                                     CONTAINER                  STATUS
  1. db-psql1                               db-psql              db                                       12.2-22:85055bd            waiting for service...

.. even if the port i correctly bound

j4care@jms1dvlvnadb01:/opt/j4care/maestro$ ss -nlt
State               Recv-Q               Send-Q                             Local Address:Port                              Peer Address:Port              Process
...
LISTEN              0                    4096                                  10.66.22.1:5432                                   0.0.0.0:*
LISTEN              0                    4096                                10.76.23.193:2376                                   0.0.0.0:*

... and works

j4care@jms1dvlvnadb01:/opt/j4care/maestro$ telnet 10.66.22.1 5432
Trying 10.66.22.1...
Connected to 10.66.22.1.
Escape character is '^]'.
petrkalina commented 4 years ago

I'm not able to reproduce now .. not sure what cause the problem .. not the lifecycle seems working!!

petrkalina commented 4 years ago 
ships:
    db:
        # address to which the containers' exposed ports bind - resolves to 10.66.22.1
        ip: db1-a.xxx
        docker_port: 2376
        # address to which the docker daemon binds
        endpoint: db1mgt-a.xxx
        tls: true
        tls_verify: true
        tls_ca_cert: /opt/j4care/conf/maestro/ship-tls/ca.crt
        tls_key: /opt/j4care/conf/maestro/ship-tls/ship.key
        tls_cert: /opt/j4care/conf/maestro/ship-tls/ship.crt
        ssl_version: PROTOCOL_TLSv1_2
services:

    db-psql:
        image:postgres
        instances:
            db-psql1:
                ship: db
                ports:
                    postgresql:
                        exposed: 5432
                        external: [ 10.66.22.1, 5432 ]
                lifecycle:
                    running: [{type: tcp, port: postgresql}]

works..