Links broken when Hydra served behind non-root reverse proxy

[jml]

I've set up my own Hydra server following the instructions in the Hydra manual, but I would like to serve it behind nginx. I have a location directive that looks like this:

location /hydra/ {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header        Host            $host;
        proxy_set_header        X-Real-IP       $remote_addr;
        proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
}

This correctly forwards to the Hydra server running on port 3000, but renders a page with broken links. They are URLs like http://myhost/queue when they should be like http://myhost/hydra/queue.

The hydra-server help doesn't show any obvious options. It suggests that I read the Catalyst manual, but it's unclear how to proceed with it—I can't find an editable configuration file in the hydra packaging.

I installed the latest hydra from the http://hydra.nixos.org/jobset/hydra/master/channel/latest channel using nix-env -i hydra and have not modified the installation.

Following advice from ikwildrpepper on IRC, I added a X-Request-Path header in the reverse proxy (proxy_set_header X-Request-Base "http://myhost/hydra";), cribbing from headcounter/deployment by @aszlig. This fixed the hyperlinks, but left the static resources broken.

(This ticket adapted from my question on StackOverflow)

[aszlig]

I'm also using URL-rewriting to strip off the /hydra prefix when the request comes back to the proxied catalyst backend.

[mschristiansen]

On Nixos there is a (symlink to a) hydra.conf file, usually in /var/lib/hydra/data/. The following two lines seem to help:

using_frontend_proxy 1
base_uri https://hydra.domain.com

For NixOS the options are described here: https://github.com/NixOS/hydra/blob/master/hydra-module.nix

Should be something similar for Nix.

[jml]

Have you verified that working under a non-root reverse proxy? My goal is not to have https://hydra.example.com/, but rather https://example.com/some-stuff/hydra/.

[mschristiansen]

I haven't tried it myself, but I would expect: base_uri https://example.com/some-stuff/hydra without the trailing slash would work in hydra.conf.

[sopvop]

Adding X-Request-Base helps. I'm not sure if base_uri in hydra.conf is used by Catalyst at all.

location /hydra/ {
    proxy_pass http://local_address/;
    proxy_redirect  http://local_address https://example.com/hydra;
    proxy_set_header        Host              $host;
    proxy_set_header        X-Real-IP         $remote_addr;
    proxy_set_header        X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header        X-Forwarded-Proto $scheme;
    proxy_set_header        X-Request-Base     /hydra;
}

This needs documenting though, I've spent half a day trying to figure out the whole perl web thing.

[tomberek]

Also seems to require X-Forwarded-Port and I've also run into issues with this check on non-root reverse proxy situations. $base ends up looking like https://domain/hydra and is longer than $referer which is just https://domain/

[cessationoftime]

I still had a lot of trouble getting nginx to serve the static hydra files such as jQuery. I eventually used this for my configuration and it solved the problem:

services.nginx.recommendedProxySettings = true;
...
...
      virtualHosts.hydra.locations = {
         "/hydra/" = {
            proxyPass = "http://hydra";
            extraConfig = ''
             proxy_redirect http://hydra http://${ip}/hydra;
             proxy_set_header  X-Request-Base    /hydra;
           '';
         };
        "/hydra/static/" = {
          extraConfig = ''
            alias ${pkgs.hydra_unstable}/libexec/hydra/root/static/;
          '';
        };

Creating a /hydra/static/ location doesn't seem to be documented anywhere.

[cessationoftime]

That was not enough to completely solve the problem. I ended up installing dnscrypt-proxy2 and then using DNS to set a hydra. prefix on the hostname. Instead of suffixing it with /hydra/