search

ilijamt / vault-plugin-secrets-gitlab

Vault Plugin for Gitlab Access Tokens
MIT License
43 stars 6 forks source link

Validate expiresAt - minimum gitlab token duration is 1 day #91

Closed TJM closed 2 months ago

TJM commented 3 months ago

If you set ttl to less than 24h, the token expiration will be set to the same day, which gives you an immediately expired token.

[tmcneely@local vault-plugin-secrets-gitlab]$ vault write gitlab/roles/project name=project-token-name path=learning-center/automation-playground/tommy-mcneely scopes="read_api" access_level=guest token_type=project ttl=4h
Key                     Value
---                     -----
access_level            guest
gitlab_revokes_token    false
name                    project-token-name
path                    learning-center/automation-playground/tommy-mcneely
role_name               project
scopes                  [read_api]
token_type              project
ttl                     4h
[tmcneely@local vault-plugin-secrets-gitlab]$ vault read gitlab/token/project
Key                Value
---                -----
lease_id           gitlab/token/project/veQt3OTxBskrwJeYJOlmDqCN
lease_duration     4h
lease_renewable    false
access_level       guest
created_at         2024-05-24T00:15:54.096Z
expires_at         2024-05-24T04:15:53.436225Z
name               vault-generated-project-access-token-dafc03f8
path               learning-center/automation-playground/tommy-mcneely
role_name          project
scopes             [read_api]
token              (CENSORED)
[tmcneely@local vault-plugin-secrets-gitlab]$ GITLAB_TOKEN=(CENSORED)
[tmcneely@local vault-plugin-secrets-gitlab]$ curl -H "PRIVATE-TOKEN: $GITLAB_TOKEN" "$GITLAB_BASE_URL/version"
{"error":"invalid_token","error_description":"Token is expired. You can either do re-authorization or token refresh."}%

Vault should handle invalidating the token at the end of its TTL, but it needs to set the expiresAt to a minimum of one day later.

ilijamt commented 3 months ago

Is this on the latest version? Let me double check as the expiration should be OK, unless Gitlab changed something in the API.

Also revolution of the token behaves differently depending on who revokes the token.

Token lifecycle is bound to your Vault token, so when it expires it will invalidate Gitlab token as well, if gitlab_revokes_token is set to false.

On Fri, 24 May 2024, 02:22 Tommy McNeely, @.***> wrote:

If you set ttl to less than 24h, the token expiration will be set to the same day, which gives you an immediately expired token.

@.*** vault-plugin-secrets-gitlab]$ vault write gitlab/roles/project name=project-token-name path=learning-center/automation-playground/tommy-mcneely scopes="read_api" access_level=guest token_type=project ttl=4h Key Value

access_level guest gitlab_revokes_token false name project-token-name path learning-center/automation-playground/tommy-mcneely role_name project scopes [read_api] token_type project ttl 4h @.*** vault-plugin-secrets-gitlab]$ vault read gitlab/token/project Key Value

lease_id gitlab/token/project/veQt3OTxBskrwJeYJOlmDqCN lease_duration 4h lease_renewable false access_level guest created_at 2024-05-24T00:15:54.096Z expires_at 2024-05-24T04:15:53.436225Z name vault-generated-project-access-token-dafc03f8 path learning-center/automation-playground/tommy-mcneely role_name project scopes [read_api] token (CENSORED) @. vault-plugin-secrets-gitlab]$ GITLAB_TOKEN=(CENSORED) @. vault-plugin-secrets-gitlab]$ curl -H "PRIVATE-TOKEN: $GITLAB_TOKEN" "$GITLAB_BASE_URL/version" {"error":"invalid_token","error_description":"Token is expired. You can either do re-authorization or token refresh."}%

Vault should handle invalidating the token at the end of its TTL, but it needs to set the expiresAt to a minimum of one day later.

— Reply to this email directly, view it on GitHub https://github.com/ilijamt/vault-plugin-secrets-gitlab/issues/91, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIWFY4FDFS2WRPHSFNRTCDZD2B4PAVCNFSM6AAAAABIGTEHDWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGMYTIMBWGA2DOOA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

TJM commented 3 months ago

We are still on 16.11.x, and I guess 17.0.x is available, but as shown above, it seems like the minimum TTL is 24 hours, which might also depend on the time of day that the token is issued, and maybe even the timezone of the gitlab server?

ilijamt commented 3 months ago

There are two modes for token expiry.

https://github.com/ilijamt/vault-plugin-secrets-gitlab?tab=readme-ov-file#ttl

I meant which version of the vault plugin are you using?

what does vault read gitlab/config say?

On Thu, 30 May 2024, 04:05 Tommy McNeely, @.***> wrote:

We are still on 16.11.x, and I guess 17.0.x is available, but as shown above, it seems like the minimum TTL is 24 hours, which might also depend on the time of day that the token is issued, and maybe even the timezone of the gitlab server?

— Reply to this email directly, view it on GitHub https://github.com/ilijamt/vault-plugin-secrets-gitlab/issues/91#issuecomment-2138539805, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIWFY42YPYJRU6KLQL4SR3ZE2CNNAVCNFSM6AAAAABIGTEHDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZYGUZTSOBQGU . You are receiving this because you commented.Message ID: @.***>

TJM commented 3 months ago

Sorry, I thought you meant gitlab version. The plugin version is v0.3.3

[tmcneely@local values]$ vault plugin list secret | grep gitlab
gitlab          v0.3.3

Config:

[tmcneely@local values]$ vault read gitlab/config
Key                          Value
---                          -----
auto_rotate_before           24h0m0s
auto_rotate_token            true
base_url                     https://gitlab.company.com
revoke_auto_rotated_token    false
token_expires_at             2025-05-21T00:00:00Z
token_id                     5719
token_sha1_hash              ba8b4a922b92523fa66180fe91d183df43011395

Reproduce:

[tmcneely@local values]$ vault read  gitlab/token/test-ttl
Key                Value
---                -----
lease_id           gitlab/token/test-ttl/UxurnT6KLBzJeD547I65CD4c
lease_duration     2h
lease_renewable    false
access_level       guest
created_at         2024-05-30T19:00:42.11Z
expires_at         2024-05-30T21:00:41.51156481Z
name               vault-generated-project-access-token-0d6b8c6b
path               learning-center/automation-playground/tommy-mcneely
role_name          test-ttl
scopes             [read_api]
token              __egx1JzGSE7zK4MyegE
[tmcneely@local values]$ curl -H "PRIVATE-TOKEN: __egx1JzGSE7zK4MyegE" $GITLAB_BASE_URL/api/v4/version
{"error":"invalid_token","error_description":"Token is expired. You can either do re-authorization or token refresh."}%
[tmcneely@local values]$ date -u
Thu May 30 19:01:43 UTC 2024
[tmcneely@local values]$ vault read  gitlab/roles/test-ttl
Key                     Value
---                     -----
access_level            guest
gitlab_revokes_token    false
name                    test-token
path                    learning-center/automation-playground/tommy-mcneely
role_name               test-ttl
scopes                  [read_api]
token_type              project
ttl                     2h

I think the problem is that Gitlab only sees "days" for token expiration, if you tried this in a couple hours, when the 2H ttl would cause the day number to change, it would probably appear to work. With older versions of gitlab, you didn't have to set the expires_at parameter, which means the way it was originally written, it probably just used a non-expiring token, but now it sends expires_at every time. So, when the expires_at is the same day, the token starts off already expired.

ilijamt commented 3 months ago

Is the time zone different between your machine and the gitlab server?

ilijamt commented 3 months ago

To calculate the TTL I do this. So this may be some issue with the TZ. Usually I always set the servers to UTC. So I will need to check to see what happens when it's in a different timezone.

Also according to https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html#when-personal-access-tokens-expire the tokens always expires at 00:00 AM UTC.

https://github.com/ilijamt/vault-plugin-secrets-gitlab/blob/aa7d6bc5777d71e8475505599d99ed3bc3322ede/utils.go#L35-L41

https://github.com/ilijamt/vault-plugin-secrets-gitlab/blob/aa7d6bc5777d71e8475505599d99ed3bc3322ede/utils_test.go#L36-L96

ilijamt commented 3 months ago

Also I see you have gitlab_revokes_token is set to false that means vault revokes the token and not Gitlab. What is your TTL for your vault token?

What does vault token lookup | grep -E '(ttl|expire)' show you?

TJM commented 3 months ago

Servers are all set to UTC, workstations are set to their local TZ, in my case US/Mountain. This is why I did date -u above, to show what the current UTC time was.

This is now several hours later, but my vault token was good for 24h.. currently:

[tmcneely@local values]$ vault token lookup | grep -E '(ttl|expire)'
creation_ttl                   24h
expire_time                    2024-05-31T18:45:42.492878298Z
explicit_max_ttl               0s
ttl                            18h48m57s

Right now, it is working, due to the "day" barrier :)

[tmcneely@local values]$ vault read  gitlab/token/test-ttl
Key                Value
---                -----
lease_id           gitlab/token/test-ttl/OWbpuuM2sxRCl9Z6HLUTMCMv
lease_duration     2h
lease_renewable    false
access_level       guest
created_at         2024-05-30T23:57:56.687Z
expires_at         2024-05-31T01:57:56.146797157Z
name               vault-generated-project-access-token-40dfd715
path               learning-center/automation-playground/tommy-mcneely
role_name          test-ttl
scopes             [read_api]
token              uKr4WxAyeHtggQaqBrsy
[tmcneely@local values]$ curl -H "PRIVATE-TOKEN: uKr4WxAyeHtggQaqBrsy" $GITLAB_BASE_URL/api/v4/version
{"version":"16.11.2-ee","revision":"d210b947e3e","kas":{"enabled":true,"externalUrl":"wss://gitlab.company.com/-/kubernetes-agent/","version":"16.11.2"},"enterprise":true}%                                                                                                                                                           [tmcneely@local values]$

We do want vault to expire the tokens, because we would like to use a TTL of 4h or less for pipeline jobs, to get around the CI_JOB_TOKEN and cross-project access changes :)

The code above is only called if "gitlabRevokesTokens" is true, since it is false, it is just using the "startTime + role.TTL" ... I propose that we pass the expiresAt parameter being sent to gitlab through calculateGitlabTTL regardless of whether gitlab is expiring the token or not.

ilijamt commented 3 months ago

Is it possible that your Vault token was near expiry, as that is the issue I used to see and the reason why gitlab_revokes_token was added. So Vault cannot revoke the token when the Vault token used to create the token expires.

TJM commented 3 months ago

No, my vault token (login token) was good for just under 24 hours the first time. The same token was still good for just under 19h when I responded the second time. This happens every time when the TTL is less than 24h (i.e 4h) and it is not later than the TTL until midnight UTC. (example with a 4h TTL, it will happen unless it is after 20:00 UTC. The problem appears to be the expires_at parameter being sent to gitlab has the current day, so gitlab considers the token already expired. NOTE that we are using GitLab 16.11.2 (as per the version query above), I don't know if this behavior changes with different versions.

ilijamt commented 3 months ago

Found the issue, you used to be able to create a token with less than 24h in Gitlab but looks like that has been fixed. So I'm working on a fix for this as well as the main token rotation.

ilijamt commented 2 months ago

Should be fixed with #97