Race condition breaks setup when oidc server is hosted behind same traefik instance

[ieugen]

Hello,

I had this setup working but I did some maintenance on the server and restarted it. Now it won't start with this message in the logs:

2024-11-06 09:37:05 [ERROR] [traefik-oidc-auth] http-get discovery endpoints - Err: Get "https://REDACTED/.well-known/openid-configuration": dial tcp 84.232.142.73:443: i/o timeout
2024-11-06 09:37:05 [ERROR] [traefik-oidc-auth] Error while retrieving discovery document: HTTP GET error
{"level":"error","entryPointName":"https","routerName":"efactura-prod-https@swarm","error":"HTTP GET error","time":"2024-11-06T09:37:05Z"}
2024-11-06 09:37:35 [ERROR] [traefik-oidc-auth] http-get discovery endpoints - Err: Get "https://REDACTED/.well-known/openid-configuration": dial tcp REDACTED:443: i/o timeout
Error while retrieving discovery document: HTTP GET error
{"level":"error","entryPointName":"https","routerName":"efactura-prod-https@swarm","error":"HTTP GET error","time":"2024-11-06T09:37:35Z"}

And in the UI

I have the OIDC app on the same node, behind traefik as well and it's working fine, I can see and authenticate to it. The issue IMO is that the plugin checks for the service before traefik has a chance to start it up and read it's swarm metadata to expose the service.

And traefik oidc does not retry to fetch config data after some time.

IMO one of these should fix:

• retry getting oidc conf, especially if it was not fetched ok the first time
• delay getting oidc configuration until first call / after traefik is fully initialized

Regards, Eugen

[sevensolutions]

Hi @ieugen thx for reporting this. Yes i think delaying the discovery query to the first request would be a good option.

[ieugen]

Thanks. IMO this should be documented as well. What happens if it fails? I think it should retry to get the document on every request, until it gets them, but I did not put much thought in this.

[ieugen]

Do you have time to fix this?

I'm kind of learning go and might take a shot at it but would need some help. My level with go is - reading the Get Started docs. I assume the configuration discovery should happen here around here https://github.com/sevensolutions/traefik-oidc-auth/blob/b6df9ced54679e683d4f91089ce2556a403524ba/main.go#L23 ?

This initialization https://github.com/sevensolutions/traefik-oidc-auth/blob/b6df9ced54679e683d4f91089ce2556a403524ba/config.go#L159 should happen in the beginning of ServeHTTP .

The code should be moved to a function so it's easy to call .

I would do it like this:

if  discoveryDoc == null  {
  grab a lock;
  if (discoveryDoc == null ) { 
    fetch discovery doc;
    finish config initialization;
  } 
  release discovery lock;
}

How can I test the setup - I have no idea how a go project works, esp a traefik plugin.

[sevensolutions]

@ieugen thx for having a closer look at this. You're very welcome if you want to try it by yourself.

Yes i think we'll move GetOidcDiscovery to the beginning of the ServeHTTP method. And as you've already written, we need to do a lock. I'am already locking when fetching the JWKS keys so you can have a look at that by searching for h.Lock.Lock().

Maybe we create a method like EnsureOidcDiscovery and call this one at the beginning of ServeHTTP. This method would then perform the lock and call GetOidcDiscovery and update this on the TraefikOidcAuth-struct.

Testing is really easy, because you don't even need to compile the Go code because it's interpreted by traefik. You can simply run docker compose up in the root directory and you should have a running instance. Just follow the steps at the end of the readme. The only thing you need is an Identity Provider. You can use ZITADEL. They provide free cloud accounts or you can self host it. You can also use Kanidm but i haven't used this by myself.

[sevensolutions]

🎉 Release 0.3.2 is out! Thx again for your contribution 👍