search

ocf / puppet

Puppet config for OCF servers and lab machines
https://www.ocf.berkeley.edu/
31 stars 71 forks source link

Proxy apphosts through death #1396

Closed Kalissaac closed 11 months ago

Kalissaac commented 11 months ago

Still need to figure out how we want to handle groups who want to switch to apphosting, since our current method of adding them to vhost-app.conf will cause their development domain to be promoted to production immediately even if they're still working on developing their app or configuring systemd. Maybe we initially add group-studentorg-berkeley-edu.apphost.ocf.berkeley.edu as their domain and then edit it to group.studentorg.berkeley.edu when they're ready to promote the site to production?

ocfjenkins[bot] commented 11 months ago

Errored hosts (0)

Changed hosts (3)

Unaffected hosts (61)

Changed hosts
diff for death.ocf.berkeley.edu, dev-death.ocf.berkeley.edu ```diff ******************************************* File[/opt/share/vhost-web.jinja] => parameters => content => @@ -18,4 +18,11 @@ # Doesn't matter too much for vhosts. RewriteRule ^(.*)$ {{vhost.redirect_dest}}$1 [L,R=302] + {% elif vhost.is_apphost %} + RewriteEngine on + RequestHeader set X-Forwarded-Proto https + ProxyPreserveHost On + SSLProxyEngine on + # Proxy to apphost server + ProxyPass / https://apphost.ocf.berkeley.edu/ upgrade=websocket {% elif vhost.disabled %} # Proxy to the local "unavailable" vhost, which serves up a friendly ******************************************* File[/usr/local/bin/build-vhosts] => parameters => content => @@ -103,4 +103,8 @@ _ @property + def is_apphost(self): + return self.bind_type == 'socket' + + @property def docroot(self): assert self.bind_type == 'docroot' @@ -387,5 +391,14 @@ if args.target == 'web': site_cfg = APACHE_SITE_CONFIG + # Build app vhosts so that they can get proxied to apphost.o.b.e + # Placed before regular vhosts so they take priority in domain matching + # (sometimes hosts have entries in both vhost.conf and vhost-app.conf) config = build_config( + get_app_vhosts(), + jinja_env.get_template('vhost-web.jinja'), + dev_config=args.dev, + ) + config += '\n\n' + config += build_config( get_vhosts(), jinja_env.get_template('vhost-web.jinja'), ******************************************* ```
diff for vampires.ocf.berkeley.edu ```diff ******************************************* File[/usr/local/bin/build-vhosts] => parameters => content => @@ -103,4 +103,8 @@ _ @property + def is_apphost(self): + return self.bind_type == 'socket' + + @property def docroot(self): assert self.bind_type == 'docroot' @@ -387,5 +391,14 @@ if args.target == 'web': site_cfg = APACHE_SITE_CONFIG + # Build app vhosts so that they can get proxied to apphost.o.b.e + # Placed before regular vhosts so they take priority in domain matching + # (sometimes hosts have entries in both vhost.conf and vhost-app.conf) config = build_config( + get_app_vhosts(), + jinja_env.get_template('vhost-web.jinja'), + dev_config=args.dev, + ) + config += '\n\n' + config += build_config( get_vhosts(), jinja_env.get_template('vhost-web.jinja'), ******************************************* ```
Unaffected hosts ``` acid.ocf.berkeley.edu anthrax.ocf.berkeley.edu arsenic.ocf.berkeley.edu asteroid.ocf.berkeley.edu autocrat.ocf.berkeley.edu avalanche.ocf.berkeley.edu bedbugs.ocf.berkeley.edu bermudatriangle.ocf.berkeley.edu bigbang.ocf.berkeley.edu biohazard.ocf.berkeley.edu blackout.ocf.berkeley.edu blight.ocf.berkeley.edu blizzard.ocf.berkeley.edu chaos.ocf.berkeley.edu corruption.ocf.berkeley.edu coup.ocf.berkeley.edu cyanide.ocf.berkeley.edu cyclone.ocf.berkeley.edu dataloss.ocf.berkeley.edu deadlock.ocf.berkeley.edu dementors.ocf.berkeley.edu democracy.ocf.berkeley.edu destruction.ocf.berkeley.edu drought.ocf.berkeley.edu fallingrocks.ocf.berkeley.edu famine.ocf.berkeley.edu firestorm.ocf.berkeley.edu firewhirl.ocf.berkeley.edu flood.ocf.berkeley.edu fraud.ocf.berkeley.edu frostbite.ocf.berkeley.edu gridlock.ocf.berkeley.edu hailstorm.ocf.berkeley.edu hal.ocf.berkeley.edu headcrash.ocf.berkeley.edu heatwave.ocf.berkeley.edu hurricane.ocf.berkeley.edu implosion.ocf.berkeley.edu invasion.ocf.berkeley.edu lethe.ocf.berkeley.edu lightning.ocf.berkeley.edu madcow.ocf.berkeley.edu maelstrom.ocf.berkeley.edu meteorstorm.ocf.berkeley.edu outbreak.ocf.berkeley.edu pestilence.ocf.berkeley.edu plague.ocf.berkeley.edu reaper.ocf.berkeley.edu riptide.ocf.berkeley.edu scurvy.ocf.berkeley.edu segfault.ocf.berkeley.edu sinkhole.ocf.berkeley.edu supernova.ocf.berkeley.edu surge.ocf.berkeley.edu thunder.ocf.berkeley.edu tsunami.ocf.berkeley.edu venom.ocf.berkeley.edu volcano.ocf.berkeley.edu whiteout.ocf.berkeley.edu wildfire.ocf.berkeley.edu windshear.ocf.berkeley.edu ```

Jenkins

oliver-ni commented 11 months ago

Probably wait on rjz approval not entirely familiar w Apache

ocfjenkins[bot] commented 11 months ago

Errored hosts (0)

Changed hosts (3)

Unaffected hosts (62)

Changed hosts
diff for death.ocf.berkeley.edu, dev-death.ocf.berkeley.edu ```diff ******************************************* File[/opt/share/vhost-web.jinja] => parameters => content => @@ -18,4 +18,10 @@ # Doesn't matter too much for vhosts. RewriteRule ^(.*)$ {{vhost.redirect_dest}}$1 [L,R=302] + {% elif vhost.is_apphost %} + RequestHeader set X-Forwarded-Proto https + ProxyPreserveHost On + SSLProxyEngine on + # Proxy to apphost server + ProxyPass / https://apphost.ocf.berkeley.edu/ upgrade=websocket {% elif vhost.disabled %} # Proxy to the local "unavailable" vhost, which serves up a friendly ******************************************* File[/usr/local/bin/build-vhosts] => parameters => content => @@ -103,4 +103,8 @@ _ @property + def is_apphost(self): + return self.bind_type == 'socket' + + @property def docroot(self): assert self.bind_type == 'docroot' @@ -387,5 +391,14 @@ if args.target == 'web': site_cfg = APACHE_SITE_CONFIG + # Build app vhosts so that they can get proxied to apphost.o.b.e + # Placed before regular vhosts so they take priority in domain matching + # (sometimes hosts have entries in both vhost.conf and vhost-app.conf) config = build_config( + get_app_vhosts(), + jinja_env.get_template('vhost-web.jinja'), + dev_config=args.dev, + ) + config += '\n\n' + config += build_config( get_vhosts(), jinja_env.get_template('vhost-web.jinja'), ******************************************* ```
diff for vampires.ocf.berkeley.edu ```diff ******************************************* File[/usr/local/bin/build-vhosts] => parameters => content => @@ -103,4 +103,8 @@ _ @property + def is_apphost(self): + return self.bind_type == 'socket' + + @property def docroot(self): assert self.bind_type == 'docroot' @@ -387,5 +391,14 @@ if args.target == 'web': site_cfg = APACHE_SITE_CONFIG + # Build app vhosts so that they can get proxied to apphost.o.b.e + # Placed before regular vhosts so they take priority in domain matching + # (sometimes hosts have entries in both vhost.conf and vhost-app.conf) config = build_config( + get_app_vhosts(), + jinja_env.get_template('vhost-web.jinja'), + dev_config=args.dev, + ) + config += '\n\n' + config += build_config( get_vhosts(), jinja_env.get_template('vhost-web.jinja'), ******************************************* ```
Unaffected hosts ``` acid.ocf.berkeley.edu anthrax.ocf.berkeley.edu arsenic.ocf.berkeley.edu asteroid.ocf.berkeley.edu autocrat.ocf.berkeley.edu avalanche.ocf.berkeley.edu bedbugs.ocf.berkeley.edu bermudatriangle.ocf.berkeley.edu bigbang.ocf.berkeley.edu biohazard.ocf.berkeley.edu blackout.ocf.berkeley.edu blight.ocf.berkeley.edu blizzard.ocf.berkeley.edu chaos.ocf.berkeley.edu corruption.ocf.berkeley.edu coup.ocf.berkeley.edu cyanide.ocf.berkeley.edu cyclone.ocf.berkeley.edu dataloss.ocf.berkeley.edu deadlock.ocf.berkeley.edu dementors.ocf.berkeley.edu democracy.ocf.berkeley.edu destruction.ocf.berkeley.edu drought.ocf.berkeley.edu fallingrocks.ocf.berkeley.edu famine.ocf.berkeley.edu firestorm.ocf.berkeley.edu firewhirl.ocf.berkeley.edu flood.ocf.berkeley.edu fraud.ocf.berkeley.edu frostbite.ocf.berkeley.edu gridlock.ocf.berkeley.edu hailstorm.ocf.berkeley.edu hal.ocf.berkeley.edu headcrash.ocf.berkeley.edu heatwave.ocf.berkeley.edu hurricane.ocf.berkeley.edu implosion.ocf.berkeley.edu invasion.ocf.berkeley.edu lethe.ocf.berkeley.edu lightning.ocf.berkeley.edu madcow.ocf.berkeley.edu maelstrom.ocf.berkeley.edu meteorstorm.ocf.berkeley.edu outbreak.ocf.berkeley.edu pestilence.ocf.berkeley.edu plague.ocf.berkeley.edu reaper.ocf.berkeley.edu riptide.ocf.berkeley.edu scurvy.ocf.berkeley.edu segfault.ocf.berkeley.edu sinkhole.ocf.berkeley.edu supernova.ocf.berkeley.edu surge.ocf.berkeley.edu thunder.ocf.berkeley.edu tornado.ocf.berkeley.edu tsunami.ocf.berkeley.edu venom.ocf.berkeley.edu volcano.ocf.berkeley.edu whiteout.ocf.berkeley.edu wildfire.ocf.berkeley.edu windshear.ocf.berkeley.edu ```

Jenkins

ocfjenkins[bot] commented 11 months ago

Errored hosts (0)

Changed hosts (3)

Unaffected hosts (62)

Changed hosts
diff for death.ocf.berkeley.edu, dev-death.ocf.berkeley.edu ```diff ******************************************* File[/opt/share/vhost-web.jinja] => parameters => content => @@ -18,4 +18,10 @@ # Doesn't matter too much for vhosts. RewriteRule ^(.*)$ {{vhost.redirect_dest}}$1 [L,R=302] + {% elif vhost.is_apphost %} + RequestHeader set X-Forwarded-Proto https + ProxyPreserveHost On + SSLProxyEngine on + # Proxy to apphost server + ProxyPass / https://apphost.ocf.berkeley.edu/ upgrade=websocket {% elif vhost.disabled %} # Proxy to the local "unavailable" vhost, which serves up a friendly ******************************************* File[/usr/local/bin/build-vhosts] => parameters => content => @@ -103,4 +103,8 @@ _ @property + def is_apphost(self): + return self.bind_type == 'socket' + + @property def docroot(self): assert self.bind_type == 'docroot' @@ -387,5 +391,14 @@ if args.target == 'web': site_cfg = APACHE_SITE_CONFIG + # Build app vhosts so that they can get proxied to apphost.o.b.e + # Placed before regular vhosts so they take priority in domain matching + # (sometimes hosts have entries in both vhost.conf and vhost-app.conf) config = build_config( + get_app_vhosts(), + jinja_env.get_template('vhost-web.jinja'), + dev_config=args.dev, + ) + config += '\n\n' + config += build_config( get_vhosts(), jinja_env.get_template('vhost-web.jinja'), ******************************************* ```
diff for vampires.ocf.berkeley.edu ```diff ******************************************* File[/opt/share/vhost-app.jinja] => parameters => content => @@ -5,4 +5,11 @@ server_name "{{vhost.fqdn}}"; _ + # Trust forwarded traffic from death + real_ip_header X-Forwarded-For; + set_real_ip_from 169.229.226.23; # death + set_real_ip_from 2607:f140:8801::1:23; + set_real_ip_from 169.229.226.37; # dev-death + set_real_ip_from 2607:f140:8801::1:37; + location /.well-known/ { alias /var/lib/lets-encrypt/.well-known/; ******************************************* File[/usr/local/bin/build-vhosts] => parameters => content => @@ -103,4 +103,8 @@ _ @property + def is_apphost(self): + return self.bind_type == 'socket' + + @property def docroot(self): assert self.bind_type == 'docroot' @@ -387,5 +391,14 @@ if args.target == 'web': site_cfg = APACHE_SITE_CONFIG + # Build app vhosts so that they can get proxied to apphost.o.b.e + # Placed before regular vhosts so they take priority in domain matching + # (sometimes hosts have entries in both vhost.conf and vhost-app.conf) config = build_config( + get_app_vhosts(), + jinja_env.get_template('vhost-web.jinja'), + dev_config=args.dev, + ) + config += '\n\n' + config += build_config( get_vhosts(), jinja_env.get_template('vhost-web.jinja'), ******************************************* ```
Unaffected hosts ``` acid.ocf.berkeley.edu anthrax.ocf.berkeley.edu arsenic.ocf.berkeley.edu asteroid.ocf.berkeley.edu autocrat.ocf.berkeley.edu avalanche.ocf.berkeley.edu bedbugs.ocf.berkeley.edu bermudatriangle.ocf.berkeley.edu bigbang.ocf.berkeley.edu biohazard.ocf.berkeley.edu blackout.ocf.berkeley.edu blight.ocf.berkeley.edu blizzard.ocf.berkeley.edu chaos.ocf.berkeley.edu corruption.ocf.berkeley.edu coup.ocf.berkeley.edu cyanide.ocf.berkeley.edu cyclone.ocf.berkeley.edu dataloss.ocf.berkeley.edu deadlock.ocf.berkeley.edu dementors.ocf.berkeley.edu democracy.ocf.berkeley.edu destruction.ocf.berkeley.edu drought.ocf.berkeley.edu fallingrocks.ocf.berkeley.edu famine.ocf.berkeley.edu firestorm.ocf.berkeley.edu firewhirl.ocf.berkeley.edu flood.ocf.berkeley.edu fraud.ocf.berkeley.edu frostbite.ocf.berkeley.edu gridlock.ocf.berkeley.edu hailstorm.ocf.berkeley.edu hal.ocf.berkeley.edu headcrash.ocf.berkeley.edu heatwave.ocf.berkeley.edu hurricane.ocf.berkeley.edu implosion.ocf.berkeley.edu invasion.ocf.berkeley.edu lethe.ocf.berkeley.edu lightning.ocf.berkeley.edu madcow.ocf.berkeley.edu maelstrom.ocf.berkeley.edu meteorstorm.ocf.berkeley.edu outbreak.ocf.berkeley.edu pestilence.ocf.berkeley.edu plague.ocf.berkeley.edu reaper.ocf.berkeley.edu riptide.ocf.berkeley.edu scurvy.ocf.berkeley.edu segfault.ocf.berkeley.edu sinkhole.ocf.berkeley.edu supernova.ocf.berkeley.edu surge.ocf.berkeley.edu thunder.ocf.berkeley.edu tornado.ocf.berkeley.edu tsunami.ocf.berkeley.edu venom.ocf.berkeley.edu volcano.ocf.berkeley.edu whiteout.ocf.berkeley.edu wildfire.ocf.berkeley.edu windshear.ocf.berkeley.edu ```

Jenkins

Kalissaac commented 11 months ago

We need to hardcode death's IPs since nginx is too old to support hostnames in set_real_ip_from :/