Vote: consider support Nginx bridge?

j1n6 commented 9 years ago

Generally consider this because Nginx offers better authentication modules.

The easy part:

bamboo has already got data for rendering template, so it result to any file format
the configuration is namespaced by haproxy, so it's realatively easy to add another nginx into the configuration schema

The tricky part:

The ZK storage needs to be migrated to avoid breaking changes
UI needs to give the option of services discovery option: nginx or haproxy

gomes commented 9 years ago

Awesome, I am using bamboo to generate both haproxy and nginx templates. I am doing without any code changes, but I am not using UI anymore.

j1n6 commented 9 years ago

That's great to know!

Can you share how you are hacking it to work? Any reason you are using both?

It would be great to know the setup and I can look into a proper solution.

Thanks in advance!

On 14 Apr 2015, at 17:29, Diogo Gomes notifications@github.com wrote:

Awesome, I am using bamboo to generate both haproxy and nginx templates. I am doing without any code changes, but I am not using UI anymore.

— Reply to this email directly or view it on GitHub https://github.com/QubitProducts/bamboo/issues/114#issuecomment-92956408.

alexanderjardim-zz commented 9 years ago

Hey,

I work with Diogo and we didn't need to hack Bamboo at all. Actually, we only use a different config file and template.

Our config file for Bamboo is:

{
  "HAProxy": {
    "TemplatePath": "/var/bamboo/config/nginx_template.conf",
    "OutputPath": "/usr/local/openresty/nginx/conf/nginx.conf",
    "ReloadCommand": "/usr/local/openresty/nginx/sbin/nginx -s reload"
  },

  "StatsD": {
    "Enabled": false,
    "Host": "localhost:8125",
    "Prefix": "bamboo-server.development."
  }
}

We use a custom nginx.conf template that Bamboo could write without any code change. Sadly, we lost dashboard management as it is obviously not compatible with Nginx. We are even putting some custom Marathon environment variables for virtual host configuration at our load balancer and reverse proxy setup.

j1n6 commented 9 years ago

Thanks for the followup. So likely I would like to make the API and data stored in ZK to be more generic for handling multiple type of export. Do you mind sharing the example nginx_template.conf (if nothing specific)?

gomes commented 9 years ago

That's right, and the motive that I opened the PR https://github.com/QubitProducts/bamboo/pull/115 it's because we need to create multiple vhosts for one marathon id

alexanderjardim-zz commented 9 years ago

We are using Bamboo in 2 cases:

Haproxy as Load balancer, SSL termination and virtual host resolver
Nginx as Reverse Proxy for dependency isolation, access control, authorization and near cache.

Beware, our nginx.conf is VERY customized and I have removed some sensitive information.

worker_processes auto;
daemon off;
error_log /usr/local/openresty/nginx/logs/error.log debug;
#worker_rlimit_nofile 65535;

events {
    use epoll;
    worker_connections  56000;
    multi_accept on;
}

http {

        send_timeout 10;
        client_body_timeout 12;
        client_header_timeout 12;
        reset_timedout_connection on;
        keepalive_timeout  15;
        #access_log off;

        sendfile on;
        tcp_nopush on;

        # Compression. Reduces the amount of data that needs to be transferred over the network
        gzip on;
        gzip_min_length 1100;
        gzip_comp_level 3;
        gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

        # Caches information about open FDs, frequently accessed files.
        # Changing this setting, in my environment, brought performance up from 560k req/sec, to 904k req/sec.
        # I recommend using some varient of these options
        open_file_cache max=5000 inactive=20s;
        open_file_cache_valid    40s;
        open_file_cache_min_uses 2;
        open_file_cache_errors   on;

        ################################

        log_format logstash '#NGINX# <$remote_addr> <$remote_user> <$connection_requests> <$time_local> '
           '<$request> <$status> <$request_time> '
           '<$body_bytes_sent> <$request_time> <$upstream_response_time> '
           '<$http_referer> <$http_user_agent>';
        access_log /usr/local/openresty/nginx/logs/access.log logstash;

        {{ $myhost := "" }}
        {{ $services := .Services }}

        {{ range $index, $app := .Apps }}

        {{ if and $app.Env.APPTENANT $app.Env.APPRESOURCE $app.Env.APPVERSION  }}

        ####### {{ $app.Env.APPTENANT }} {{ $app.Env.APPRESOURCE }} {{ $app.Env.APPVERSION }} EXTERNAL ######
        upstream {{ $app.Env.APPTENANT }}-{{ $app.Env.APPRESOURCE }}-{{ $app.Env.APPVERSION }}-cluster-external {
                {{ if $myhost }}
                {{ range $page, $task := .Tasks }}
                {{ if eq $task.Host $myhost }}
                server 172.17.42.1:{{ $task.Port }} weight=1000000;
                {{ else }}
                server {{ $task.Host }}:{{ $task.Port }};
                {{ end }}
                {{ end }}
                {{ else }}
                {{ range $page, $task := .Tasks }}
                server {{ $task.Host }}:{{ $task.Port }};
                {{ end }}
                {{ end }}
        }

        ####### {{ $app.Env.APPTENANT }} {{ $app.Env.APPRESOURCE }} {{ $app.Env.APPVERSION }} INTERNAL ######
        upstream {{ $app.Env.APPTENANT }}-{{ $app.Env.APPRESOURCE }}-{{ $app.Env.APPVERSION }}-cluster-internal {
                {{ if $myhost }}
                {{ range $page, $task := .Tasks }}
                {{ if eq $task.Host $myhost }}
                server 172.17.42.1:8080 weight=1000000;
                {{ else }}
                server {{ $task.Host }}:8080;
                {{ end }}
                {{ end }}
                {{ else }}
                {{ range $page, $task := .Tasks }}
                server {{ $task.Host }}:8080;
                {{ end }}
                {{ end }}
        }

        {{ end }}

        {{ end }}

        ##### EXTERNAL ####
        server {

                listen 8080;

                # Nginx request monitor
                location /nginx_status {
                    stub_status on;
                    access_log   off;
                    allow all;

                }

                {{ range $index, $app := .Apps }}
                {{ if and $app.Env.APPTENANT $app.Env.APPRESOURCE $app.Env.APPVERSION  }}

                ####### {{ $app.Env.APPTENANT }} {{ $app.Env.APPRESOURCE }} {{ $app.Env.APPVERSION }} EXTERNAL ######

                {{ if or $app.Env.APPPROXY400 $app.Env.APPPROXY401 $app.Env.APPPROXY403 $app.Env.APPPROXY404 $app.Env.APPPROXY405 $app.Env.APPPROXY415 $app.Env.APPPROXY422 $app.Env.APPPROXY500 $app.Env.APPPROXY502 $app.Env.APPPROXY503 $app.Env.APPPROXY504 }}
                location ~ ^/error/({{ $app.Env.APPTENANT }})/({{ $app.Env.APPRESOURCE }})/({{ $app.Env.APPVERSION }})/(.*) {
                    root /usr/local/openresty/nginx/html;
                }
                {{ end }}

                location ~ ^/({{ $app.Env.APPTENANT }})/({{ $app.Env.APPRESOURCE }})/({{ $app.Env.APPVERSION }})(.*) {
                        set $xbrand $1;
                        set $xresource $2;
                        set $xversion $3;
                        set $xuri $4;
                        set $xurl $xuri;
                        {{ if not $app.Env.APPNOAUTH }}#access_by_lua_file "/usr/local/openresty/access/access.lua";{{ end }}
                        {{ if $app.Env.APPPROXY400 }}error_page 400 /error/{{ $app.Env.APPTENANT }}/{{ $app.Env.APPRESOURCE }}/{{ $app.Env.APPVERSION }}/400.html;{{ end }}
                        {{ if $app.Env.APPPROXY401 }}error_page 401 /error/{{ $app.Env.APPTENANT }}/{{ $app.Env.APPRESOURCE }}/{{ $app.Env.APPVERSION }}/401.html;{{ end }}
                        {{ if $app.Env.APPPROXY403 }}error_page 403 /error/{{ $app.Env.APPTENANT }}/{{ $app.Env.APPRESOURCE }}/{{ $app.Env.APPVERSION }}/403.html;{{ end }}
                        {{ if $app.Env.APPPROXY404 }}error_page 404 /error/{{ $app.Env.APPTENANT }}/{{ $app.Env.APPRESOURCE }}/{{ $app.Env.APPVERSION }}/404.html;{{ end }}
                        {{ if $app.Env.APPPROXY405 }}error_page 405 /error/{{ $app.Env.APPTENANT }}/{{ $app.Env.APPRESOURCE }}/{{ $app.Env.APPVERSION }}/405.html;{{ end }}
                        {{ if $app.Env.APPPROXY415 }}error_page 415 /error/{{ $app.Env.APPTENANT }}/{{ $app.Env.APPRESOURCE }}/{{ $app.Env.APPVERSION }}/415.html;{{ end }}
                        {{ if $app.Env.APPPROXY422 }}error_page 422 /error/{{ $app.Env.APPTENANT }}/{{ $app.Env.APPRESOURCE }}/{{ $app.Env.APPVERSION }}/422.html;{{ end }}
                        {{ if $app.Env.APPPROXY500 }}error_page 500 /error/{{ $app.Env.APPTENANT }}/{{ $app.Env.APPRESOURCE }}/{{ $app.Env.APPVERSION }}/500.html;{{ end }}
                        {{ if $app.Env.APPPROXY502 }}error_page 502 /error/{{ $app.Env.APPTENANT }}/{{ $app.Env.APPRESOURCE }}/{{ $app.Env.APPVERSION }}/502.html;{{ end }}
                        {{ if $app.Env.APPPROXY503 }}error_page 503 /error/{{ $app.Env.APPTENANT }}/{{ $app.Env.APPRESOURCE }}/{{ $app.Env.APPVERSION }}/503.html;{{ end }}
                        {{ if $app.Env.APPPROXY504 }}error_page 504 /error/{{ $app.Env.APPTENANT }}/{{ $app.Env.APPRESOURCE }}/{{ $app.Env.APPVERSION }}/504.html;{{ end }}                        
                        proxy_intercept_errors on;
                        proxy_cache off;
                        proxy_set_header  X-Real-IP  $remote_addr;
                        proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header  Host $http_host;
                        proxy_redirect          off;
                        proxy_next_upstream     error timeout invalid_header http_500;
                        proxy_connect_timeout   2;
                        rewrite ^/(.*)$ $xurl break;
                        add_header 'X-Upstream-Proxy' $server_addr;
                        proxy_pass http://{{ $app.Env.APPTENANT }}-{{ $app.Env.APPRESOURCE }}-{{ $app.Env.APPVERSION }}-cluster-external;
                }

                {{ end }}
                {{ end }}

                location / {
                    return 501;
                }

        }

        ##### INTERNAL ####
        server {

                listen 80;

                {{ range $index, $app := .Apps }}
                {{ if and $app.Env.APPTENANT $app.Env.APPRESOURCE $app.Env.APPVERSION  }}

                ####### {{ $app.Env.APPTENANT }} {{ $app.Env.APPRESOURCE }} {{ $app.Env.APPVERSION }} EXTERNAL ######

                location ~ ^/({{ $app.Env.APPTENANT }})/({{ $app.Env.APPRESOURCE }})/({{ $app.Env.APPVERSION }})(.*) {
                        set $xbrand $1;
                        set $xresource $2;
                        set $xversion $3;
                        set $xuri $4;

                        set $xurl $xuri;
                        proxy_cache off;
                        proxy_set_header  X-Real-IP  $remote_addr;
                        proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
                        proxy_set_header  Host $http_host;
                        proxy_redirect          off;
                        proxy_next_upstream     error timeout invalid_header http_500;
                        proxy_connect_timeout   2;
                        add_header 'X-Upstream-Proxy' $server_addr;
                        proxy_pass http://{{ $app.Env.APPTENANT }}-{{ $app.Env.APPRESOURCE }}-{{ $app.Env.APPVERSION }}-cluster-internal;
                }
                {{ end }}
                {{ end }}

                    location / {
                    return 501;
                }

    }
}

QubitProducts / bamboo

Vote: consider support Nginx bridge? #114