Enabling Direct Communication Between API and Dex Services Inside Kubernetes Clusters

[Bart-vanDongen]

When configuring terrakube, there might be scenarios where you need the API service and the Dex authentication services to establish direct communication. This setup is particularly beneficial when operating in environments protected by IP whitelisting or behind Web Application Firewalls (WAF).

To facilitate this, an additional configuration parameter can be introduced to support internal communication between these services. Should this parameter be absent in the Helm configuration, the system should revert to using the default URI settings for connectivity. This ensures seamless interaction between services without routing traffic outside of the Kubernetes cluster.

example:

security:
  useOpenLDAP: true
  adminGroup: "TERRAKUBE_ADMIN"
  patSecret: "bAUaAojZP3XhkuE2rWBtR3gRAHPzQKkx"
  internalSecret: "AxxPdgpCi72f8WhMXCTGhtfMRp6AuBfj"
  dexClientId: "example-app"
  dexClientScope: "email openid profile offline_access groups"
  dexIssuerUriExternal: "http://terrakube-api.minikube.net/dex"
  dexIssuerUriInternal: "http://terrakube-dex.svc.cluster.local/"
  existingSecret: false
  ldapConfigSecretName: terrakube-openldap-secrets

[alfespa17]

Hello @Bart-vanDongen

The value that is being used when deploying the helm chart is .dex.config.issuer in the API configuration here and inside the registry configuration here

I could try to use dexIssuerUriInternal: "http://terrakube-dex.svc.cluster.local/" for the API/Registry but I will have to test if that works in this part of the code because the dex token will be issued by http://terrakube-api.minikube.net/dex but internally validate with http://terrakube-dex.svc.cluster.local/

https://github.com/AzBuilder/terrakube/blob/843b3bf7dadb3a7c2c36429dc91b34593220b045/api/src/main/java/org/terrakube/api/plugin/security/authentication/dex/DexAuthenticationManagerResolver.java#L69

I will do some test and update this issue, thank you for the suggestion.

[alfespa17]

I did some test @Bart-vanDongen

And it looks like the token issuer should match to the parameter that we send to the api/registry configuration, if the values does not match you get the following error:

java.lang.IllegalStateException: The Issuer "https://5556-azbuilder-terrakube-qctju4md51d.ws-us110.gitpod.io/dex" provided in the configuration did not match the requested issuer "http://localhost:5556/dex"
        at org.springframework.util.Assert.state(Assert.java:97) ~[spring-core-5.3.31.jar:5.3.31]
        at org.springframework.security.oauth2.jwt.JwtDecoderProviderConfigurationUtils.validateIssuer(JwtDecoderProviderConfigurationUtils.java:84) ~[spring-security-oauth2-jose-5.7.11.jar:5.7.11]

The reason is this validation in spring security

https://github.com/spring-projects/spring-security/blob/ce54a6db182479d071bf65e151a58edbcb3b2db2/oauth2/oauth2-jose/src/main/java/org/springframework/security/oauth2/jwt/JwtDecoderProviderConfigurationUtils.java#L87

[Bart-vanDongen]

Hi @alfespa17. Thank you for quickly looking into this. Could that not be solved by setting 2 uri. One for the issuer and one for the connection.

See github issue in spring-security/oauth here they are talking about the same setup: https://github.com/spring-projects/spring-security/issues/11515#issuecomment-1204372465

This is the github issue where they added the option to set authorization-uri and token-uri property instead of the issuer-uri property. For example, if the internal network traffic must be routed through a Proxy, you can bypass discovery by configuring the this.

https://github.com/spring-projects/spring-security/issues/8882#user-content-client-registrations

[alfespa17]

Hi @alfespa17. Thank you for quickly looking into this. Could that not be solved by setting 2 uri. One for the issuer and one for the connection.

See github issue in spring-security/oauth here they are talking about the same setup: spring-projects/spring-security#11515 (comment)

This is the github issue where they added the option to set authorization-uri and token-uri property instead of the issuer-uri property. For example, if the internal network traffic must be routed through a Proxy, you can bypass discovery by configuring the this.

spring-projects/spring-security#8882 (comment)

This could work, I will do some test, thank you for the suggestion

[alfespa17]

Hello @Bart-vanDongen

I checked the proposed solution using NimbusJwtDecoder.withIssuerLocation but that method was added in spring security 6.1.0 according to this pull request

https://github.com/spring-projects/spring-security/issues/10309

We are using spring-boot-security 2.7.18 which include spring-security 5.7.11

https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-security/2.7.18

So we will have to update spring-boot to version 3, I was planning to do that next month so I will leave this issue open until I complete the migration for now

[alfespa17]

By the way I will move the issue to the main repository

[alfespa17]

I have been testing the above suggestion using spring security but it does not work, there is an open issue related to that configuration here