feat: Support service account token for argocd server authentication

[ChichiCaleb]

Summary

Support service account token for argocd server authentication

Motivation

• Argocd application and applicationset are already considered highlevel abstractions, however end-users might want to put together argocd offered capabilities into a more simplified interface either as part of an IDP implementation or even for personal convenience

• This requires setting up and authenticating with argocd server incluster, however the current implementation works well with out of cluster programmatic authentication.

• For incluster setup a typical flow should be that all authentication and authorization be done incluster

• The typical way to achieve this as of now , requires using admin user with initial argocd password.

Proposal

Proposed implementation; Use kubernetes service account token together with admin user.

Expiration tokens can be generated off service account incluster and use for argocd server authentication.

Since the current kubernetes service account token generation is secure and can easily be rotated.

The user can  periodically rotate this token , use it for argocd authentication, ensuring simplicity, and security, in addition leveraging kubernetes native implementation of authentication

Update VerifyUsernamePassword function to authenticate with service account token

// Allow service account token authentication only for the 'admin' user
    if username == "admin" && mgr.isKubernetesToken(password) {
        // Simply verify that the token is valid
        valid, err := mgr.verifyKubernetesToken(password,kubeClientset)
        if err != nil || !valid {
            mgr.updateFailureCount(username, true)
            return InvalidLoginErr
        }
    } else {
        // If it's not a token or the username isn't 'admin', proceed with standard password verification
        valid, _ := passwordutil.VerifyPassword(password, account.PasswordHash)
        if !valid {
            mgr.updateFailureCount(username, true)
            return InvalidLoginErr
        }
    }

Usage

sa := &corev1.ServiceAccount{
        ObjectMeta: metav1.ObjectMeta{
            Name:      "test-sa",
            Namespace: "default",
        },
    }
    createdSA, err := kubeClientset.CoreV1().ServiceAccounts("default").Create(context.TODO(), sa, metav1.CreateOptions{})

###########################################################

tokenRequest := &authv1.TokenRequest{
        Spec: authv1.TokenRequestSpec{
            Audiences:         []string{"https://kubernetes.default.svc.cluster.local"},
            ExpirationSeconds: int64Ptr(3600), // Token valid for 1 hour
        },
    }
    tokenResponse, err := kubeClientset.CoreV1().ServiceAccounts("default").CreateToken(context.TODO(), createdSA.Name, tokenRequest, metav1.CreateOptions{})

    password = tokenResponse.Status.Token

A scheduleTokenRefresh() function can be setup by the user in another go routine to periodically refresh token and client

Link to PR

feat: Support service account token for argocd server authentication

[jannfis]

I would like to understand more of the use cases and reasoning behind this.

Is it possible that you join the Argo CD contributor's meeting (each Thursday, 11:15am eastern time - more info here).

Since changes to authentication are a very sensitive matter, this might require a full fledged proposal and definitely more discussion.

[ChichiCaleb]

@jannfis that will be great

[torfjor]

Wouldn't the proposed solution let any KSA authenticate as the admin user?

I think a more general solution would be to expose dex' token exchange endpoints and provide a way to map external identities to argo cd machine users.

[ChichiCaleb]

The audiences field in the TokenReviewSpec specifies the intended audience of the token. The audience helps ensure that the token is intended for the specific API server or service.

In this case the cluster in which argocd is running, so basically the authentication will be valid for incluster authentication.

If the service acct token is generated from another cluster it will fail authentication

Hope I'm not getting things wrong, atleast thats what I assume to be the typical flow

[torfjor]

Sure, but that would also let a Pod running as evil/attacker present its KSA token and be able to authenticate as admin.

You would at least have to do some assertions on the claims under "kubernetes.io", or the sub field (system:serviceaccount:namespace:ksa) matching what you expect in the configuration.

[ChichiCaleb]

I got your logic. however the administrator should adequately protect their cluster. I could add BoundObjectRef:

  // Verify that the token is bound to a specific pod/service account
    if resp.Status.BoundObjectRef != nil {
        if resp.Status.BoundObjectRef.Kind != "Pod" {
            log.Printf("Token is not bound to a pod. BoundObjectRef: %v", resp.Status.BoundObjectRef)
            return false, nil
        }
        if resp.Status.BoundObjectRef.Name != "my-pod" || resp.Status.BoundObjectRef.Namespace != "expected-namespace" {
            log.Printf("Token is bound to an unexpected pod: %v", resp.Status.BoundObjectRef)
            return false, nil
        }
        log.Printf("Token is bound to pod: %v/%v", resp.Status.BoundObjectRef.Namespace, resp.Status.BoundObjectRef.Name)
    } else {
        log.Printf("Token is not bound to any object.")
        return false, nil
    }

and users implementation will be something like this;

  tokenRequest := &v1.TokenRequest{
        Spec: v1.TokenRequestSpec{
            Audiences: []string{"https://kubernetes.default.svc.cluster.local"},
            ExpirationSeconds: int64Ptr(3600), // Optional: set expiration time (in seconds)
            BoundObjectRef: &v1.BoundObjectReference{
                Kind:       "Pod",
                APIVersion: "v1",
                Name:       "my-pod",       // Replace with your pod name
                UID:        "your-pod-uid", // Replace with your pod UID
            },
        },
    }

stiil a poorly protected cluster is still prone to many attack.

even with current implementation of initial argocd admin password. it is still prone to such attack, except the user wishes to totally disable admin priviledges. for which we all know that this admin priviledges are sometimes needed for such programmatic access

[torfjor]

Bound object references in service account tokens gives the creator of the token a way to have kube-api programmatically validate that the referenced object still exists when receiving a token review request. Eg. "make sure my token is no longer valid when my Pod is removed from the API". It is not suited for making assertions about the identity carried with the KSA token, as the embedded attenuations are controlled by the creator of the token.

Kubernetes is a multi-tenant environment. It is not common for argo cd installations to be an island where you can trust your neighboring namespaces.

The implementation would have to at minimum include a configurable mapping of namespace/service account names pairs that are allowed to present themselves as the argocd admin user.

I think the scope of this implementation is too narrow. In-cluster authentication is nice, but there are multiple use cases where being able to present a valid OIDC ID token from any issuer that argo cd's configured OIDC provider trusts would come in handy. An example would be a CI/CD job that could present its system-issued ID token to argo cd's OIDC provider and in exchange get back a token that would let it assume the identity of an argo cd machine user. This process is defined in RFC 8693 and already implemented by the bundled IdP provider, dex.

[ChichiCaleb]

If OIDC provider is used ,Don't you think we are going back to the same issue of multiple level of token request, how is that different from directly requesting token from argocd server with your predefined privileges. The incluster implementation wishes to eliminate such external request of token, rather everything done in cluster, which is the typical flow of most incluster implementations

[ChichiCaleb]

A typical example can be seen with kubernetes incluster authentication when setting up a controller no need to pass kubeconfig, even with terraform, the kubernetes provider kubeconfig field is left empty

[torfjor]

If OIDC provider is used ,Don't you think we are going back to the same issue of multiple level of token request, how is that different from directly requesting token from argocd server with your predefined privileges. The incluster implementation wishes to eliminate such external request of token, rather everything done in cluster, which is the typical flow of most incluster implementations

It's different because you provide your own set of ambient credentials, which are provided by the platform you are running on. These credentials can be independently verified by your IdP, and their mapped identities can be looked up in configuration. No static or long-lived credentials are required.

Technically, with the OIDC approach, you could add the local cluster issuer to your IdP RP configuration and you would be able to exchange local Kubernetes service account tokens for a token that could be used for impersonating a argocd machine user.

[ChichiCaleb]

If I got your flow right

1. Generate service account token
2. Send to OIDC provider
3. Get a verified token in return
4. Use the token to authenticate to argocd server via OIDC endpoint

[torfjor]

Yes. Either generate a KSA token or use a projected volume (recommended as the kubelet will automatically rotate the token in the projected volume before it expires).

Some pointers:

• https://dexidp.io/docs/guides/token-exchange/ Dex docs on token exchange
• https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/ GitLab docs on workload identity in CI. Good examples of using curl to do the token exchange dance according to the RFC spec

[ChichiCaleb]

Does dex OIDC accept kubernetes service account token

I'm not familiar with Dex OIDC, just curious

[torfjor]

Its token exchange endpoint will accept a KSA token, granted that the issuer of that token exposes an OIDC discovery document (/.well-known/openid-configuration) and that issuer has been configured as a ~provider~ connector in Dex.

kube-api serves the discovery document and the cluster JWKS by default, but you may have to fiddle with the ClusterRoleBindig system:service-account-issuer-discovery and make sure that unauthenticated users are bound.

[ChichiCaleb]

Nice insight @torfjor Will read more around this

[ChichiCaleb]

Tried figuring a way to implement using dex, required a lot of logic at first glance. later came up with this , hopefully it serves same purpose in ensuring reliability and security

1. User passes the trusted service account to argocd server args

   server:
   extraArgs:
   - "--in-cluster-service-account=system:serviceaccount:default:specific-sa"

2. Refactor verifyKubernetesToken

command.Flags().StringVar(&inclusterServiceAccount, "in-cluster-service-account", env.StringFromEnv("ARGOCD_IN_CLUSTER_SERVICE_ACCOUNT", ""), "In-Cluster Kubernetes Service Account")

type SessionManager struct {
    inclusterServiceAccount string
}
###########################################################################
func (mgr *SessionManager) verifyKubernetesToken(token string, kubeClientset kubernetes.Interface) (bool, error) {
    tokenReview := &authenticationv1.TokenReview{
        Spec: authenticationv1.TokenReviewSpec{
            Token: token,
            Audiences: []string{
                "https://kubernetes.default.svc.cluster.local",
            },
        },
    }

    resp, err := kubeClientset.AuthenticationV1().TokenReviews().Create(context.TODO(), tokenReview, metav1.CreateOptions{})
    if err != nil {
       return false, fmt.Errorf("Error during TokenReview creation: %v", err)
    }

    if !resp.Status.Authenticated {
       return false, fmt.Errorf("Token is not authenticated: %v", resp.Status.Error)
    }

    claims, err := decodeToken(token)
    if err != nil {
        return false, fmt.Errorf("error decoding token: %v", err)
    }

    if claims["sub"] != mgr.inclusterServiceAccount {
        return false, fmt.Errorf("token subject does not match expected service account: %v", claims["sub"])
    }

    return true, nil
}