Open philnickl opened 5 months ago
Hi, I got the exact same issue, same config.
Appears the issue is already noted here #8956
Had the same issue an needed to swap to version 2023.10 to get it working. With trace logging turned on 2024.2.2 doesn't appear to log anything related to trying to find the app when hitting /output.authentik.io/auth/nginx
it just logs the headers (showing a host that should match my app) then immediately issues the 404 without ever indicating whether it tried to do anything, no app ended up associated with the log output.
I have the same issue with 2024.2. It works just fine on 2023.10.7.
please provide the logs of your server container
When accessing the embedded outpost the only log row I see in the server container is:
{"auth_via": "unauthenticated", "domain_url": "redacted.com", "event": "/outpost.goauthentik.io/auth/nginx", "host": "redacted.com", "level": "info", "logger": "authentik.asgi", "method": "GET", "pid": 1753928, "remote": "10.40.33.202", "request_id": "8a56ee9c20644100a3722f22e3899dcd", "runtime": 11, "schema_name": "public", "scheme": "http", "status": 404, "timestamp": "2024-05-16T08:18:14.785086", "user": "", "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"}
This is indeed a problem for me on a clean install
For me the following made it work again after almost eating my keyboard from frustration:
after doing so it worked fine for me, hope this helps you.
It's been a while since I've last did stuff with the forward auth stuff but I don't think I remember that this was required earlier
Same here - @HeavyTobi they are till in the "Selected Applications" -> moving around does not help
The problem appears to only occur when using single application mode instead of domain mode (which currently breaks redirects).
Sharing that I followed the fixed Nginx configuration template for forward auth in #10181. Forward auth is working for me now. The embedded outpost no longer returns a 404.
embedded outpost 404 is breaking me as well on clean install. @BeryJu any recommendations or things to know when using the embedded outpost with Docker? Been debugging for many hours
this is not a question rather a bug, /outpost.goauthentik.io/ping is returning 404 with not even a single line of log version: 2024.6.3 @philnickl
@engAmirEng I got something working this morning, maybe I can write something up to show end to end setup. The logs were a bit sparse, but instead of using the embedded outpost in core, I deployed a standalone outpost and it seems to be working. Debugging was a tad better because AUTHENTIK_DEBUG
to "true"
got me a bit further.
I am getting 404s too. If I delete the embedded outpost and restart my container it's recreated.
Then I get this when I try https://auth.domain/outpost.goauthentik.io
As soon as I modify the outpost to include the application I get a 404
upgraded to the latest version that doesn't require a db update, I still get this
@andres-asm Are you running embedded outpost or external outpost? Have you added debug logs to nginx?
Add this to your http
block.
log_format detailed_headers_log '[$time_local] $remote_addr - $remote_user - $server_name to: $upstream_addr: $request $status '
'Host: "$host" '
'X-Real-IP: "$remote_addr" '
'X-Forwarded-For: "$proxy_add_x_forwarded_for" '
'X-Forwarded-Proto: "$scheme" '
'X-Forwarded-Host: "$host" '
'X-Forwarded-Port: "$server_port" '
'X-Original-URL: "$scheme://$http_host$request_uri" '
'http_host: "$http_host" '
'request_uri: "$request_uri" '
'cookie: "$http_cookie" '
'set_cookie: "$sent_http_set_cookie" '
'upstream_http_set_cookie: "$upstream_http_set_cookie"';
In each location block, you can get detailed debug logs by dropping in:
access_log /var/log/nginx/authentik_access_debug.log detailed_headers_log;
You can debug these directly with tail -f /var/log/nginx/authentik_access_debug.log
Ok, got this working using embedded outpost with NGINX on latest version Notes: https://ryan-schachte.com/notes/authentik_embedded_outpost_docker/
@andres-asm & @engAmirEng let me know if this helps you as I spent a few hours debugging to get this working. Got manual outpost and embedded working fine with latest version.
The documentation could use some work in this area.
@Schachte my problem is authentik.myhost.domain/outpost.goauthentik.io/auth/nginx retuning 404, what did you exactly solved about this?
@Schachte my problem is authentik.myhost.domain/outpost.goauthentik.io/auth/nginx retuning 404, what did you exactly solved about this?
Did you review the NGINX and Docker config from my link? You most likely have a misconfiguration as I was getting that previously as well. I documented the changes that resolved it for me.
@Schachte the issue is with the embedded outpost. I actually added another outpost to the same docker compose file and got everything working with that other outpost.
I tried on a default docker setup, as soon as I select an app for the embedded outpost it starts giving a 404 like @engAmirEng said.
I can try the debug stuff later.
This is my docker-compose.yml
services:
postgresql:
image: docker.io/library/postgres:12-alpine
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
start_period: 20s
interval: 30s
retries: 5
timeout: 5s
volumes:
- database:/var/lib/postgresql/data
environment:
POSTGRES_PASSWORD: ${PG_PASS:?database password required}
POSTGRES_USER: ${PG_USER:-authentik}
POSTGRES_DB: ${PG_DB:-authentik}
env_file:
- .env
redis:
image: docker.io/library/redis:alpine
command: --save 60 1 --loglevel warning
restart: unless-stopped
healthcheck:
test: ["CMD-SHELL", "redis-cli ping | grep PONG"]
start_period: 20s
interval: 30s
retries: 5
timeout: 3s
volumes:
- redis:/data
server:
image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2024.2.4}
restart: unless-stopped
command: server
environment:
AUTHENTIK_REDIS__HOST: redis
AUTHENTIK_POSTGRESQL__HOST: postgresql
AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
AUTHENTIK_POSTGRESQL__NAME: ${PG_DB:-authentik}
AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
volumes:
- ./data/media:/media
- ./data/templates:/templates
env_file:
- .env
ports:
- "${AUTHENTIK_PORT_HTTP:-9001}:9000"
- "${AUTHENTIK_PORT_HTTPS:-9443}:9443"
depends_on:
- postgresql
- redis
worker:
image: ${AUTHENTIK_IMAGE:-ghcr.io/goauthentik/server}:${AUTHENTIK_TAG:-2024.2.4}
restart: unless-stopped
command: worker
environment:
AUTHENTIK_REDIS__HOST: redis
AUTHENTIK_POSTGRESQL__HOST: postgresql
AUTHENTIK_POSTGRESQL__USER: ${PG_USER:-authentik}
AUTHENTIK_POSTGRESQL__NAME: ${PG_DB:-authentik}
AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS}
# `user: root` and the docker socket volume are optional.
# See more for the docker socket integration here:
# https://goauthentik.io/docs/outposts/integrations/docker
# Removing `user: root` also prevents the worker from fixing the permissions
# on the mounted folders, so when removing this make sure the folders have the correct UID/GID
# (1000:1000 by default)
user: root
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- ./data/media:/media
- ./data/certs:/certs
- ./data/templates:/templates
env_file:
- .env
depends_on:
- postgresql
- redis
authentik_proxy:
image: ghcr.io/goauthentik/proxy
# Optionally specify which networks the container should be
# might be needed to reach the core authentik server
# networks:
# - foo
ports:
- 9009:9009
- 9449:9443
environment:
AUTHENTIK_HOST: https://redacted
AUTHENTIK_INSECURE: "false"
AUTHENTIK_TOKEN: redacted
# Starting with 2021.9, you can optionally set this too
# when authentik_host for internal communication doesn't match the public URL
# AUTHENTIK_HOST_BROWSER: https://external-domain.tld
volumes:
database:
driver: local
redis:
driver: local
@andres-asm I'm using the embedded outpost just fine with the latest version. Please review my NGINX configuration and compare to yours. Note the headers. Maybe also post the UI config you have for the embedded outpost.
I got the manual outpost working as well, but the reverse proxy configuration between the two differs significantly (and it's not obvious). The docs are of course wrong.
To hopefully add some clarification to this:
This is not a bug, this is intended functionality. Previously, the /outpost.goauthentik.io
paths would be available under all hostnames, not just the ones configured for the proxy providers. This meant that it was possible to go to the domain you access authentik under and to the path /outpost.goauthentik.io/start
and start authentication and get an incorrect cookie. It would also happen quite frequently that with an incorrectly setup reverse proxy, that the reverse proxy would redirect to that invalid URL, which would cause redirect loops.
Hence since a couple releases, those URLs are only available when using the correct Host
header to make sure this does not happen. Hence when accessing the URL directly in the browser on the authentik domain you see a 404 error, and when curling the /ping
url you also get a 404 endpoint.
@andres-asm @engAmirEng
I've updated my blog to walk through k8s setup. Working for me fine - https://ryan-schachte.com/notes/authentik_embedded_outpost_docker/
I've never used Authentik and was trying to configure it today. I think this is what's causing me confusion because I want to put Authenik on a dedicated (VM) host (auth.example.com
) and have other hosts use it for auth. I'm not sure the docs take the behavior described here into account.
This is not a bug, this is intended functionality. Previously, the
/outpost.goauthentik.io
paths would be available under all hostnames, not just the ones configured for the proxy providers. This meant that it was possible to go to the domain you access authentik under and to the path/outpost.goauthentik.io/start
and start authentication and get an incorrect cookie. It would also happen quite frequently that with an incorrectly setup reverse proxy, that the reverse proxy would redirect to that invalid URL, which would cause redirect loops.Hence since a couple releases, those URLs are only available when using the correct
Host
header to make sure this does not happen. Hence when accessing the URL directly in the browser on the authentik domain you see a 404 error, and when curling the/ping
url you also get a 404 endpoint.
Does that mean these docs are out of date?
If you're trying to proxy to an upstream over HTTPS, you need to set the Host header to the value they expect for it to work correctly.
reverse_proxy /outpost.goauthentik.io/* https://outpost.company {
header_up Host {http.reverse_proxy.upstream.hostport}
}
I don't understand how that config would work anyway because it's saying to set the Host
header to the upstream host for half of the proxied requests while leaving the Host
as-is for the other half, isn't it? For now, assume that header_up Host
would get set on both the reverse_proxy
and forward_auth
(docs), which is just an opinionated version of reverse_proxy
. I think the config that page is describing would look like this:
app1.example.com {
tls
# Always forward outpost path to actual outpost
reverse_proxy /outpost.goauthentik.io/* https://auth.example.com {
# Host set to auth.example.com
header_up Host {upstream_hostport}
}
# Forward authentication to outpost
forward_auth https://auth.example.com {
uri /outpost.goauthentik.io/auth/caddy
header_up Host {upstream_hostport}
copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Email X-Authentik-Name X-Authentik-Uid X-Authentik-Jwt X-Authentik-Meta-Jwks X-Authentik-Meta-Outpost X-Authentik-Meta-Provider X-Authentik-Meta-App X-Authentik-Meta-Version
trusted_proxies 192.0.2.1
}
reverse_proxy app1:80
}
If that's the case, I would only have to terminate TLS and proxy traffic for auth.example.com
on my Authentik host:
Based on what @BeryJu said is the new behavior, you would not set the Host
header because it would need to match the domain for the application in Authentik.
app1.example.com {
tls
# Always forward outpost path to actual outpost
# Host defaults to app1.example.com
reverse_proxy /outpost.goauthentik.io/* https://auth.example.com
# Forward authentication to outpost
forward_auth https://auth.example.com {
uri /outpost.goauthentik.io/auth/caddy
copy_headers X-Authentik-Username X-Authentik-Groups X-Authentik-Email X-Authentik-Name X-Authentik-Uid X-Authentik-Jwt X-Authentik-Meta-Jwks X-Authentik-Meta-Outpost X-Authentik-Meta-Provider X-Authentik-Meta-App X-Authentik-Meta-Version
trusted_proxies 192.0.2.1
}
reverse_proxy app1:80
}
That would mean I need to do TLS termination for every domain on my Authentik host, right?
Is that correct?
To hopefully add some clarification to this:
This is not a bug, this is intended functionality. Previously, the
/outpost.goauthentik.io
paths would be available under all hostnames, not just the ones configured for the proxy providers. This meant that it was possible to go to the domain you access authentik under and to the path/outpost.goauthentik.io/start
and start authentication and get an incorrect cookie. It would also happen quite frequently that with an incorrectly setup reverse proxy, that the reverse proxy would redirect to that invalid URL, which would cause redirect loops.Hence since a couple releases, those URLs are only available when using the correct
Host
header to make sure this does not happen. Hence when accessing the URL directly in the browser on the authentik domain you see a 404 error, and when curling the/ping
url you also get a 404 endpoint.
what do you mean it's a feature????? then what is this? https://docs.goauthentik.io/docs/troubleshooting/forward_auth/general#ensure-outpostgoauthentikio-is-accessible
if they do not fix their doc there wont be a future for the project
@BeryJu - I would agree the docs here are painfully convoluted and wasted many hours collectively between many folks in this thread, including myself.
For me the following made it work again after almost eating my keyboard from frustration:
- go to Admin Area --> Applications --> Outposts --> authentik Embedded
- Actions -> Edit
- make sure your application is in the selected applications list
![]()
after doing so it worked fine for me, hope this helps you.
It's been a while since I've last did stuff with the forward auth stuff but I don't think I remember that this was required earlier
It works for my caddy forward-auth! Thank you very much!
I have been unable to configure the embedded outpost to work with NGINX Forward Auth. Curl requests when pinging the outpost return a 404 (Authentik 404 page, not a NGINX 404 page). SAML/oAuth2 Applications already configured in Authentik continue to work fine.
Accessing the endpoint service returns a 500 NGINX error - due to authentik returning a 404 in the auth-url I believe.
2024/04/03 21:17:05 [error] 27#27: *3494 auth request unexpected status: 404 while sending to client, client: 10.0.1.141
The Embedded Outpost config appears to be correct