WIP: Better implementation for template

[ekozan]

I think Bamboo need to work automatically with the environment variables and with more than one bridge ( haproxy, nginx, iptables )

• Bamboo_Enable=y -> can be omited ( usefull for disable all the app )
• Bamboo_Haproxy_enable=y -> can be omited
• Bamboo_Haproxy_http=8000 -> can be omited
• Bamboo_Haproxy_https=8001 -> default disabled
• Bamboo_Haproxy_tcp_8000=8002 -> optional
• Bamboo_Iptables_enable=y -> default no
• Bamboo_Iptables_udp_8000=8003-> optional

What do you think about that? in any case i'll work on this problems for my needs

PR will come fast

[j1n6]

Agree, this is definitely a good path. Probably related to #114 ? Do you have some sort of solution that could make existing API more generic?

[ekozan]

Yes totally it's will close this issue

First we need to refactor all directory of this project ... it's little tricky

For me the Templates class is a Tools not a Service same for the execCommand let's separate this in Tools folder ( why not of the same ocasion remove Main dir who is really ugly same for the qzk dir )

For the config we need to remove acl and setup JSON/Yaml or other config and it's override by Env variables

The event handlers are only allowed to handle and dispatch the events the events "APP UPDATE" need to be send to all registered "bridge" with apps config. who will carry all the config and service reboot job

the per app config can look like

{
    "haproxy": {
        "enabled": true,
        "http_port": "8080",
        "acl": "htr(host) -i blabla.com",
        "tcp_ports": {
            "81": "8081",
            "82": "8082"
        }
    },
    "iptables": {
        "enabled": true,
        "udp_ports": {
            "83": "8083"
        }
    }
}

[ekozan]

maybe it close #84 too

[j1n6]

Ah, you are using iptables for UDP. I like your solution. Based on what you suggested, I restructured a bit so that it ensures we can configure the order of those services configuration (e.g. ACL may have priority):

{ 
   // each app may have multiple services
   // this is at Marathon App level
    "services": [{
        // Get servicePort information from marathon apps API
        "servicePort": 8080,
        // Get protocol information from Marathon API, default to HTTP
        "protocol": "tcp",
        // support a number of bridges
        "bridges": {
            // backwards compatible  (auto migrate the ZK data to this format)
            "haproxy": {
                "enabled": true,
                "port": 443,
                "acl": "htr(host) -i blabla.com"
            },
            // if we are going to implement a specific handler?
            "nginx": {
                "enabled": true,
                "port": 443,
                "serverName": "blabla.com"
            },
           // something I would like to support
            "consul": {
                "enabled": true
            },
           // something I would like to support
           "etcd": {
                "enabled": true
           }
        }
    }, {
        "servicePort": 9090,
        "protocol": "udp",
        "bridges": {
            "iptables": {
                "enabled": true,
                "port": 8083
            },
            "nginx": {
                "enabled": false,
                "custom_key": "value"
            },
            "custom": {
                "enabled": false,
                "custom_key": "value"
            }
        }
    }]
}

@hyper-carrot @MiLk @mbabineau Any comments on the multiple services port and type of protocol support?

@gomes @alexanderjardim Any comments on the Nginx bridge support?

[j1n6]

@veverjak @xiaods I think you guys were asking for better multiple services port support too, so it would be nice to get some opinions.

[j1n6]

While I'm working on #109 for the next release, it would be good to find a stable API so that it unlocks a lot of feature requests. Thanks for all the feedbacks!

[xiaods]

I am have to say, come on, this is amazing things. :+1:

[hyper0x]

It's so cool! I see that you shift the core of configuration from single proxy to services. That is a significant and great change.

[jwerak]

@activars ad JSON you posted earlier, I am little confused, where would you specify this service logic?

Can Marathon app have such info in itself?

[MiLk]

Your proposal only affect templateData.Services and not templateData.Apps, right?

It doesn't change anything regarding the support of multiple service ports.

I should be able to specify one ACL for each service port my app have.

[ekozan]

No my proposal is for add the possibility if multi port and multi bridge in the same app !

[MiLk]

You should not have to specify any port in Bamboo, Marathon gives you everything.

[ekozan]

it's not true marathon not give to you the outside port !

for me it's better to setup per service config

{
    "haproxy": {
        "enabled": true,
        "ports": [
            {"type":"http","port":"80", "marathon_port":"8082","acl": "htr(host) -i blabla.com"},
            {"type":"tcp","port":"81", "marathon_port":"8082"},
            {"type":"tcp","port":"82", "marathon_port":"8081"}
        ]
    },
    "iptables": {
        "enabled": true,
        "udp_ports": {
            "83": "8083"
        }
    }
}

[hyper0x]

I like @ekozan 's proposal!

[MiLk]

You have the outside port chosen by Marathon in https://mesosphere.github.io/marathon/docs/rest-api.html#get-/v2/apps and in App.ServicePort.

Look at the example provided in the marathon repo.

[ekozan]

the service Port is Unique per App so for example if you set 2 app with the same service port {"message":"Requested service port 9000 conflicts with a service port in app /web/authGate"}

so it's can be an good way for avoid port collision for tcp and udp but not for http/https

[j1n6]

Meanwhile, this is the new v0.2.11 release. https://github.com/QubitProducts/bamboo/releases/tag/v0.2.11

Unless there are major bugs, I shall focus on the next release for longer term multiple services and multiple bridge support.

I am preparing a list of requirement for addressing this API change, so we have a common ground to discuss possible implementation.

[nickpoorman]

I would change "enabled" to "public". The issue becomes do we want haproxy routing to to the WAN, while we might still want internal (private network) services to be able to communicate. Where Ingress from WAN, "public": false" gets sent to 403 backend or something.

[xiaods]

any update?

[lclarkmichalek]

I think #180 actually helps with this a lot. Not sure it goes as far as you might like, but it has features that seem to help with your use case here.