Caddy+Portainer behind oauth2

[xmflsct]

How can I deploy Portainer on Caddy behind oauth2.

Portainer is running under subdomain, like portainer.domain.com. When trying to login, seems like javascript is throwing error, being redirected to auth, although I have authenticated. Portainer runs fine under localhost. Any suggestions? Thanks!

[magikstm]

Could you please share:

• The error you are getting (in javascript)
• Your full caddyfile
• The full URL of the page on which you login

[xmflsct]

The errors I am getting in browser are: /#/auth:1 Access to XMLHttpRequest at 'https://auth.server.domain.com/login?backTo=https%3A%2F%2Fportainer.server.domain.com%2Fapi%2Fextensions%3Fstore%3Dfalse' (redirected from 'https://portainer.server.domain.com/api/extensions?store=false') from origin 'https://portainer.server.domain.com' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. vendor.82dd1cbf3e3da3500b77.js:61 GET https://auth.server.domain.com/login?backTo=https%3A%2F%2Fportainer.server.domain.com%2Fapi%2Fextensions%3Fstore%3Dfalse net::ERR_FAILED and Access to XMLHttpRequest at 'https://auth.server.domain.com/login?backTo=https%3A%2F%2Fportainer.server.domain.com%2Fapi%2Fendpoints%3Flimit%3D100%26start%3D0' (redirected from 'https://portainer.server.domain.com/api/endpoints?limit=100&start=0') from origin 'https://portainer.server.domain.com' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. vendor.82dd1cbf3e3da3500b77.js:61 GET https://auth.server.domain.com/login?backTo=https%3A%2F%2Fportainer.server.domain.com%2Fapi%2Fendpoints%3Flimit%3D100%26start%3D0 net::ERR_FAILED

My full Caddyfile is:

*.server.domain.com {
  tls me@domain.com {
    dns cloudflare
  }
}

auth.server.domain.com {
  tls {
    wildcard
  }
  redir 302 {
    if {path} is /
    / /login
  }
  login {
    google client_id={$GOOGLE_OAUTH_CLIENT_ID},client_secret={$GOOGLE_OAUTH_CLIENT_SECRET}
    redirect_check_referer false
    redirect_host_file /etc/redirect_hosts.txt
    cookie_domain server.domain.com
  }
}

(int-auth) {
  jwt {
    path /
    redirect https://auth.server.domain.com/login?backTo=https%3A%2F%2F{host}{rewrite_uri_escaped}
    allow sub myemail@gmail.com
  }
}

portainer.server.domain.com {
  import int-auth
  tls {
    wildcard
  }
  gzip
  proxy / portainer:9000 {
    websocket
    transparent
  }
}

And I tried to remove the import int-auth from within the block, everything works perfectly. Thanks!

[magikstm]

You seem to be having CORS issues.

I believe you could either: 1) Fix these in your code 2) Try using the CORS plugin

I would suggest trying to use the CORS plugin first and see if it fixes it.

I would add the plugin to your Caddy then add that to your config: cors

As such:

*.server.domain.com {
cors  
tls me@domain.com {
    dns cloudflare
  }
}

If it does fix it, you could either refine its use or adapt your javascript code to make it work.

Ref:

• https://github.com/captncraig/cors/blob/master/README.md
• https://stackoverflow.com/questions/35588699/response-to-preflight-request-doesnt-pass-access-control-check/43110816
• https://www.w3.org/wiki/CORS_Enabled

[xmflsct]

Thanks! But no luck still.

Portainer is pre-packed so I could not easily modify the source code.

And I tried to add the CORS plugin, placing it in multiple blocks but still get the same error. Could you help me understand, why those resources requested by JS are still being redirected to auth server although I have successfully signed in via oauth2?

[magikstm]

Do you have Caddy logs and errors activated?

Are there more info in them?

I'm unfortunately not familiar with Portainer.

I did find this though: https://github.com/portainer/portainer/issues/1939

I would need more details to investigate further. I would have though the CORS plugin would solve this.

Did you ensure you:

1. Replaced the executable of Caddy (which includes the plugin)
2. Changed the configfile
3. Restarted Caddy for changes to take effect
4. Closed the browser fully to ensure cache didn't get in the way

Try setting CORS for all domains just in case (it could be tweaked afterwards) by adding that at the top of your caddyfile:

http://, https:// {
CORS
}

[xmflsct]

Still a nope. Yes I have restarted the service every time I update Caddyfile. :) And since I am using auth, I also clear site related cookie so that it starts from fresh (including tried a different browser).

Herewith the log I get so far with Caddy. It does not seem to print more detailed message. I get both log stdout and errors stdout from within the container.

caddy | i.p.i.p - - [16/Sep/2019:11:34:30 +0000] "GET / HTTP/1.1" 303 119 caddy | i.p.i.p - - [16/Sep/2019:11:34:30 +0000] "GET /login?backTo=https%3A%2F%2Fportainer.server.domain.com%2F HTTP/1.1" 200 1714 caddy | i.p.i.p - - [16/Sep/2019:11:34:33 +0000] "GET /login/google HTTP/1.1" 302 0 This is when I was redirected to Google auth.

caddy | time="2019-09-16T11:34:39Z" level=info msg="successfully authenticated" type=application username="myemail@gmail.com" caddy | i.p.i.p - - [16/Sep/2019:11:34:39 +0000] "GET /login/google?state=xxx&code=xxx&scope=email+profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.profile+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email&authuser=0&session_state=xxx&prompt=none HTTP/1.1" 303 0 caddy | i.p.i.p - - [16/Sep/2019:11:34:39 +0000] "GET / HTTP/1.1" 200 3279 This is when I was successfully authenticated with Google.

caddy | i.p.i.p - - [16/Sep/2019:11:34:39 +0000] "GET /api/status HTTP/1.1" 200 104 caddy | i.p.i.p - - [16/Sep/2019:11:34:39 +0000] "GET /api/settings/public HTTP/1.1" 200 218 caddy | i.p.i.p - - [16/Sep/2019:11:34:40 +0000] "GET /api/users/admin/check HTTP/1.1" 204 0 caddy | i.p.i.p - - [16/Sep/2019:11:34:40 +0000] "GET /api/settings/public HTTP/1.1" 200 218 This is when Portainer's own auth page is fully loaded.

caddy | i.p.i.p - - [16/Sep/2019:11:35:11 +0000] "POST /api/auth HTTP/1.1" 200 463 I tried to login through Portainer obviously. :)

caddy | i.p.i.p - - [16/Sep/2019:11:35:11 +0000] "GET /api/extensions?store=false HTTP/1.1" 303 142 caddy | i.p.i.p - - [16/Sep/2019:11:35:11 +0000] "OPTIONS /login?backTo=https%3A%2F%2Fportainer.server.domain.com%2Fapi%2Fextensions%3Fstore%3Dfalse HTTP/1.1" 200 0 caddy | i.p.i.p - - [16/Sep/2019:11:35:11 +0000] "GET /api/endpoints?limit=100&start=0 HTTP/1.1" 303 152 caddy | i.p.i.p - - [16/Sep/2019:11:35:11 +0000] "OPTIONS /login?backTo=https%3A%2F%2Fportainer.server.domain.com%2Fapi%2Fendpoints%3Flimit%3D100%26start%3D0 HTTP/1.1" 200 0 This is when things go wrong. I don't know why those calls are redirected by to auth server again which of course failed to load its resources.

Any ideas?

[magikstm]

Why does portainer.server.domain.com have import int-auth and auth.server.domain.com doesn't?

I believe that may be the issue. Try adding that import to that one as well or moving it to *.server.domain.com.

Also, that part seems unneeded:

redir 302 {
    if {path} is /
    / /login
  }

[xmflsct]

Still nope.

Now my Caddyfile looks like this. Just as brief, details are the same as before.

(int-auth) ## Initialise dynamic redirect back

*.server.domain.com ## Set up wildcard certificate to use later

auth.server.domain.com ## Import (int-auth), set up Google thingys

portainer.server.domain.com ## Import (int-auth), the actual reverse proxy

Same error in the log, that HTTP OPTIONS are always being directed to auth. GET and POST are both fine. Very interestingly, if I import int-auth only within *.server.domain.com, no authentication will happen at all. I have place it each time within a subdomain to let it work. Could be that the problem is on Portainer's side, but I am not that into JS.

[xmflsct]

@magikstm After Googling and Googling, I found the problem, https://caddy.community/t/oauth-proxy-rejecting-some-requests/4546/2.

[magikstm]

How did you resolve it?

[xmflsct]

Like said in the link, double proxy to rewrite header. Not a neat solution. :) I was trying to see if I can change jwt's header type since login uses cookie. But if I add token_source header, I cannot get pass jwt anymore. So I gave up and use double proxy.

But actually it still has problem sometimes, don't know why. I sometimes get kicked out of Portainer, and one of the calls get 401.

At the end, I find out that there is a pretty hidden argument to turn off Portainer's authentication! So now my setup is fully running, without double proxy, without Portainer's authentication, but instead oauth2 by login+jwt. :)

[magikstm]

Awesome. :)