Closed ken5scal closed 2 years ago
Hi @ken5scal
Can you try setting access_token_lifetime
to 1800s
? I'm wondering if there's some clock skew...
I had the same problem and tried setting access_token_lifetime
to 1800s
but it didn't solve.
@sethvargo @fujikky
Actually, it worked!
Just to note, I was required to set one or more Google Workspace related API scopes to access_token_scopes
.
- id: google-auth-3
name: 'Authenticate to Google Cloud'
uses: google-github-actions/auth@v0
with:
token_format: 'access_token'
workload_identity_provider: 'projects/{PROJECT_ID}/locations/global/workloadIdentityPools/${ID_POOL}/providers/${PROVIDER}'
service_account: 'sa@example.com'
access_token_scopes: 'https://www.googleapis.com/auth/admin.reports.audit.readonly'
access_token_subject: 'gws@gws.com'
access_token_lifetime: '1800s'
I have retrieved following credential as result of google-github-actions/auth
. How do we use this to access Google Workspace API in our code?
{
"type": "external_account",
"audience": "//iam.googleapis.com/projects/${PROJECT_ID}/locations/global/workloadIdentityPools/${IDENTITY_POOL_ID}/providers/${PROVIDER_ID}",
"subject_token_type": "urn:ietf:params:oauth:token type:jwt",
"token_url": "https://sts.googleapis.com/v1/token",
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/sa@${PROJECT_ID}.iam.gserviceaccount.com:generateAccessToken",
"credential_source": {
"url": "https://pipelines.actions.githubusercontent.com/SOME_RANDOM_STRING/00000000-0000-0000-0000-000000000000/_apis/distributedtask/hubs/Actions/plans/SOME_RANDOM_STRING/idtoken?api-version=2.0&audience=https%3A%2F%2Fiam.googleapis.com%2Fprojects%${PROJECT_ID}%2Flocations%2Fglobal%2FworkloadIdentityPools%2F${IDENTITY_POOL_ID}%2Fproviders%2F${PROVIDER_ID}",
"headers": {
"Authorization": "***"
},
"format": {
"type": "json",
"subject_token_field_name": "value"
}
}
}
Using this value just like the way we do in service account key
, the API returned an authorization error with following messages (depends on how you implement):
googleapi: Error 401: Access denied. You are not authorized to read activity records., authError
google: could not parse JSON key: google: read JWT from JSON credentials: 'type' field is "external_account" (expected "service_account")
@ken5scal
Thanks for the info!
However, adding access_token_scopes
did not solve the problem.
Here is the YAML of the Actions I tried.
- id: auth
uses: google-github-actions/auth@v0
with:
workload_identity_provider: projects/00000000000/locations/global/workloadIdentityPools/github-actions/providers/foo-provider
service_account: foo@bar-project-id.iam.gserviceaccount.com
token_format: access_token
access_token_scopes: https://www.googleapis.com/auth/spreadsheets
access_token_subject: user@example.com
access_token_lifetime: 1800s
I got the same error message.
An access token subject was specified, triggering Domain-Wide Delegation flow. This flow does not support specifying an access token lifetime of greater than 1 hour.
Error: google-github-actions/auth failed with: failed to sign JWT using foo@bar-project-id.iam.gserviceaccount.com: {
"error": {
"code": 403,
"message": "The caller does not have permission",
"status": "PERMISSION_DENIED"
}
}
@ken5scal what API(s) are you trying to call after authenticating? The action.yml in the original issue stops at the auth step. Note all technologies support WIF (for example, bq and gsutil do not support WIF).
@fujikky make sure you've granted roles/iam.serviceAccountTokenCreator
and roles/iam.workloadIdentityUser
to the external identity ("principalSet").
@sethvargo Thanks. I'm trying to use Admin API to list activities for Google Workspace applications such as the Admin console application or the Google Drive application. More specifically I am using Admin Go Package. https://pkg.go.dev/google.golang.org/api/admin/reports/v1#ActivitiesService.List
I believe the value in scope (
https://www.googleapis.com/auth/admin.reports.audit.readonly
) is correct since non-keyless domain-wide delegation is working.
@sethvargo Thanks for the reply! As you said, my identity's principalSet was missing a policy. So I attached the policy with the following command. But I'm still getting a PERMISSION_DENIED error.
gcloud iam service-accounts add-iam-policy-binding "${SA_EMAIL}" \
--project="${PROJECT_ID}" \
--role="roles/iam.serviceAccountTokenCreator" \
--member="principalSet://iam.googleapis.com/${WORKLOAD_IDENTITY_POOL_ID}/attribute.repository/${REPO}"
gcloud iam service-accounts add-iam-policy-binding "${SA_EMAIL}" \
--project="${PROJECT_ID}" \
--role="roles/iam.workloadIdentityUser" \
--member="principalSet://iam.googleapis.com/${WORKLOAD_IDENTITY_POOL_ID}/attribute.repository/${REPO}"
I have confirmed that the policy is attached in the Cloud Console. Are there any other possible factors?
And the error message is still "failed to sign JWT using ..."? Do you have debug logs you could share?
failed to sign JWT using ...
That's right. Let me share the debug log later
@sethvargo
Thanks for the help! My current error message is not about JWT. Here is the debug log.
@sethvargo It finally worked! 🎉
First, I founded that the attribute.repository
was missing in the provider's attributes! I redid the README steps and set the correct attributes. Sorry for my mistake.
Next, I found a bug in the auth action.
The code to suppress the warning, fixed 2 days ago, does not seem to pass through the buildDomainWideDelegationJWT
if the access_token_lifetime
exceeds 3600 seconds.
https://github.com/google-github-actions/auth/blob/714f1fe243ca012171c3f4b3ec7d205bcb3589c9/src/main.ts#L213
I changed to the previous version uses: google-github-actions/auth@v0.7.1
in my actions yaml, and the impersonated access token was worked successfully.
@sethvargo
I think we misunderstood each other.
What I was saying is that google-github-actions/auth@v0
does return a credential file Created credentials file at "/home/runner/work/xxx/xxx/gha-creds-e3360af22c9dc486.json"
and store it under GOOGLE_APPLICATION_CREDENTIALS
.
Now, I use a Golang App to fetch the audit log...and get an error.
panic: failed to listing activities: googleapi: Error 401: Access denied. You are not authorized to read activity records., authError
@fujikky fixed in https://github.com/google-github-actions/auth/pull/178 and will be released as 0.7.3. Thanks for catching that, and I'm glad to see this is working for you now.
@ken5scal which client library is that, and which version are you using? In general, you should never need to parse GOOGLE_APPLICATION_CREDENTIALS
, since all the official google SDKs automatically look for that environment variable and use it for authentication. reports.NewService(ctx)
should "just work"
@sethvargo Thanks, I'm using https://pkg.go.dev/google.golang.org/api@v0.80.0 which is the latest and official one.
I actually changed my code not to parse GOOGLE_APPLICATION_CREDENTIALS
, but still giving me googleapi: Error 401: Access denied. You are not authorized to read activity records., authError
That library doesn't support WIF yet: https://github.com/googleapis/google-api-go-client/issues/750
Ohhh.... Thanks, I would watch the issue in there @sethvargo
I am also struggling with this. I am using the Python googleapiclient
library to upload files to Google Drive using both GCP's Workload Identity Provider (WIF) and delegation to my Google Workspace user. The delegation seems to fail and instead, the files are uploaded as the service account (to which I already gave access to the folder), but it then gets stuck later on since it does not have sufficient storage space in it's Google Drive allowance. How can I get this work?
Here's a sample of my GitHub workflow:
permissions:
id-token: write # This is required for requesting the JWT
contents: read # This is required for actions/checkout
jobs:
Deploy:
runs-on: ubuntu-latest-m
steps:
- name: Git clone the repository
uses: actions/checkout@v4
- name: Google auth
id: 'auth'
uses: google-github-actions/auth@v2
with:
token_format: 'access_token'
workload_identity_provider: '${{ secrets.GCP_WIF_PROVIDER }}'
service_account: '${{ secrets.GOOGLE_DRIVE_SERVICE_ACCOUNT }}'
access_token_lifetime: 1800s
access_token_scopes: https://www.googleapis.com/auth/drive
access_token_subject: '${{ vars.GOOGLE_DRIVE_SUBJECT_ACCOUNT }}'
delegates: '${{ secrets.GOOGLE_DRIVE_SERVICE_ACCOUNT }}'
- name: Print the credentials file path
run: |
echo "Credentials file path: ${{ steps.auth.outputs.credentials_file_path }}"
cat ${{ steps.auth.outputs.credentials_file_path }}
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.x'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install google-auth google-auth-oauthlib google-auth-httplib2 google-api-python-client
- name: Upload to Google Drive
env:
GOOGLE_DRIVE_FOLDER_ID: ${{ secrets.GOOGLE_DRIVE_FOLDER_ID }}
GOOGLE_APPLICATION_CREDENTIALS: ${{ steps.auth.outputs.credentials_file_path }}
run: |
python .github/workflows/upload_to_gdrive.py my-folder target-folder-location
and the contents of upload_to_gdrive.py
:
import logging
import os
from pathlib import Path
import sys
from googleapiclient.discovery import build
from google.auth import default
from googleapiclient.http import MediaFileUpload
from googleapiclient.errors import HttpError
BATCH_SIZE = 10 # Adjust this value based on your needs
SKIPPED_FOLDERS = {".git"}
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
def create_folder_if_not_exists(service, folder_name: str, parent_id: str) -> str:
query = f"name='{folder_name}' and '{parent_id}' in parents and mimeType='application/vnd.google-apps.folder' and trashed=false"
results = (
service.files().list(q=query, spaces="drive", fields="files(id)").execute()
)
if results["files"]:
folder_id = results["files"][0]["id"]
logger.info(f"Folder '{folder_name}' found with ID: {folder_id}")
return folder_id
else:
folder_metadata = {
"name": folder_name,
"mimeType": "application/vnd.google-apps.folder",
"parents": [parent_id],
}
folder = service.files().create(body=folder_metadata, fields="id").execute()
logger.info(
f"Folder '{folder_name}' created successfully with ID: {folder.get('id')}"
)
return folder.get("id")
def upload_file(service, file_path: str, parent: tuple[str, str]):
parent_name, parent_id = parent
file_metadata = {"name": Path(file_path).name, "parents": [parent_id]}
media = MediaFileUpload(file_path, resumable=True)
try:
file = (
service.files()
.create(body=file_metadata, media_body=media, fields="id")
.execute()
)
logger.debug(
f"File '{file_path}' uploaded successfully with ID: {file.get('id')} to folder with name: {parent_name} with ID: {parent_id}"
)
return file.get("id")
except HttpError as error:
logger.error(f"An error occurred while uploading '{file_path}': {error}")
raise error
def upload_folder(
service, folder_path: Path, parent_id: str, target_subfolder: str | None = None
):
target_parent_id = (
create_folder_if_not_exists(service, target_subfolder, parent_id)
if target_subfolder
else parent_id
)
folder_path_to_id = {
Path("."): target_parent_id
} # Caching the root folder as the parent ID
for root, dirs, files in folder_path.walk():
if any(folder in str(root) for folder in SKIPPED_FOLDERS):
continue
# Cache folder paths to prevent redundant folder creation
for dir_name in dirs:
dir_path = Path(root) / dir_name
relative_path = dir_path.relative_to(folder_path)
if relative_path not in folder_path_to_id:
folder_id = create_folder_if_not_exists(service, dir_name, parent_id)
folder_path_to_id[relative_path] = folder_id
# Process files in batches
file_batch = []
for file in files:
file_path = root / file
rel_folder_path = file_path.parent.relative_to(folder_path)
folder_id = folder_path_to_id.get(rel_folder_path, parent_id)
file_batch.append((file_path, (rel_folder_path, folder_id)))
if len(file_batch) == BATCH_SIZE:
upload_batch(service, file_batch)
file_batch = []
if file_batch:
upload_batch(service, file_batch)
def upload_batch(service, file_batch: list[tuple[str, tuple[str, str]]]):
for file_path, (parent_name, parent_id) in file_batch:
upload_file(service, file_path, (parent_name, parent_id))
def main(folder_path: str, target_subfolder: str | None = None):
credentials, project = default()
service = build("drive", "v3", credentials=credentials)
folder_id = os.environ.get("GOOGLE_DRIVE_FOLDER_ID")
if not folder_id:
raise ValueError("GOOGLE_DRIVE_FOLDER_ID environment variable is not set")
upload_folder(service, Path(folder_path), folder_id, target_subfolder)
if __name__ == "__main__":
if len(sys.argv) < 2:
logger.warn("Usage: python upload_to_drive.py <folder_path>")
sys.exit(1)
folder_path = sys.argv[1]
target_subfolder = sys.argv[2] if len(sys.argv) == 3 else None
if not Path(folder_path).is_dir():
logger.error(f"Error: '{folder_path}' is not a valid directory.")
sys.exit(1)
main(folder_path, target_subfolder)
TL;DR
credentials_json
runs successfully, I believe this issue is specific to Domain-Wide Delegation.roles/iam.serviceAccountTokenCreator
androles/iam.workloadIdentityUser
Expected behavior
GitHub Action is able to retrieve an access tokens created for Domain-Wide Delegation.
Observed behavior
An access token subject was specified, triggering Domain-Wide Delegation flow. This flow does not support specifying an access token lifetime of greater than 1 hour. Error: google-github-actions/auth failed with: failed to sign JWT using gws-access@${{PROJECT_ID}}.iam.gserviceaccount.com: { "error": { "code": 403, "message": "The caller does not have permission", "status": "PERMISSION_DENIED" } }
Action YAML
Log output
Additional information
No response