google-github-actions / upload-cloud-storage

A GitHub Action for uploading files to a Google Cloud Storage (GCS) bucket.
https://cloud.google.com/storage
Apache License 2.0
224 stars 50 forks source link

Action does not allow uploading single file to a bucket path #309

Closed pratheekrebala closed 1 year ago

pratheekrebala commented 1 year ago

TL;DR

I am unable to use this action to upload a single file to a given path in a bucket. It seems like the package only allows specifying a prefix and not a explicit destination path.

E.g. I am unable to achieve the following: /tmp/data.yaml => gs://bucket/.acl/data.yaml

If I try to run the action with the following variables:

uses: google-github-actions/upload-cloud-storage@v1
with:
  path: /tmp/acl.yaml
  destination: bucket/.acl/acl.yaml
  parent: false
  gzip: false
  headers: |-
    content-type: text/yaml

The file is uploaded to: gs://bucket/.acl/acl.yaml/acl.yaml instead of gs://bucket/.acl/acl.yaml

Expected behavior

I would expect that a single file would be uploaded to: gs://bucket/.acl/acl.yaml

Observed behavior

A file is instead uploaded to: gs://bucket/.acl/acl.yaml/acl.yaml

Action YAML

name: upload-gcs
on: [workflow_call]
jobs:
  upload:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write
    steps:
    - id: gh-checkout
      uses: actions/checkout@v3
    - id: setup-node
      uses: actions/setup-node@v3

    - id: setup-dependencies
      run: |
        npm install yaml

    - id: parse-acl
      uses: actions/github-script@v6
      with:
        script: |
          // script generates a YAML file with the ACL policy and saves it to /tmp/acl.yaml
          // script also outputs a "acl" output variable with the path to the generated file (/tmp/acl.yaml)

    - id: gcp-authenticate
      uses: google-github-actions/auth@v1
      with:
        workload_identity_provider: [identityProvider] # redacted
        service_account: [serviceAccount] # redacted

    - id: gcp-upload-acl
      uses: google-github-actions/upload-cloud-storage@v1
      with:
        path: ${{ steps.parse-acl.outputs.acl }} # value is /tmp/acl.yaml
        destination: gs://[bucket]/.acl/acl.yaml # bucket name redacted
        parent: false
        gzip: false
        headers: |-
          content-type: text/yaml

Log output

# I am unable to provide the entire log output here, but here's the log specific to this action.

##[debug]Evaluating condition for step: 'Run google-github-actions/upload-cloud-storage@v1'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Run google-github-actions/upload-cloud-storage@v1
##[debug]Loading inputs
##[debug]Evaluating: steps.parse-acl.outputs.acl
##[debug]Evaluating Index:
##[debug]..Evaluating Index:
##[debug]....Evaluating Index:
##[debug]......Evaluating steps:
##[debug]......=> Object
##[debug]......Evaluating String:
##[debug]......=> 'parse-acl'
##[debug]....=> Object
##[debug]....Evaluating String:
##[debug]....=> 'outputs'
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'acl'
##[debug]=> '/tmp/acl.yaml'
##[debug]Result: '/tmp/acl.yaml'
Run google-github-actions/upload-cloud-storage@v1
  with:
    path: /tmp/acl.yaml
    destination: [bucket]/.acl/acl.yaml
    parent: false
    gzip: false
    headers:
      content-type: text/yaml
    resumable: true
    concurrency: 100
    process_gcloudignore: true
  env:
    CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE: [redacted]
    GOOGLE_APPLICATION_CREDENTIALS: [redacted]
    GOOGLE_GHA_CREDS_PATH: [redacted]
    CLOUDSDK_CORE_PROJECT: [redacted]
    CLOUDSDK_PROJECT: [redacted]
    GCLOUD_PROJECT: [redacted]
    GCP_PROJECT: [redacted]
    GOOGLE_CLOUD_PROJECT: [redacted]
##[debug]Computed absoluteRoot from "/tmp/acl.yaml" to "/tmp" (isDir: false)
##[debug]Computed computedGlob from "" to "acl.yaml"
##[debug]Found 1 files: ["acl.yaml"]
##[debug]Processing gcloudignore
##[debug]Using .gcloudignore at: /home/runner/work/test-gh-upload/test-gh-upload/.gcloudignore
##[debug]Parsed ignore list: [".git","gha-creds-*.json",".github"]
##[debug]Uploading 1 files: ["acl.yaml"]
##[debug]Computed bucket as "[bucket]"
##[debug]Computed prefix as ".acl/acl.yaml"
::group::Upload files
Upload files
  Uploading /tmp/acl.yaml to gs://[bucket]/.acl/acl.yaml/acl.yaml
  ##[debug]Uploading: {"destination":"[bucket]/.acl/acl.yaml/acl.yaml","metadata":{"contentType":"text/yaml"},"gzip":false,"resumable":true,"configPath":[redacted],"ts":[redacted],"source":"/tmp/acl.yaml"}
  ::endgroup::
##[debug]Node Action run completed with exit code 0
##[debug]Set output uploaded = .acl/acl.yaml/acl.yaml
##[debug]Finishing: Run google-github-actions/upload-cloud-storage@v1

Additional information

This also prevents uploading a static file with a different name. E.g. /tmp/acl.yaml => gs://bucket/.acl/github.yaml

I think the only way to make this work would be:

mkdir -p /tmp/acl
echo "tktktk" > /tmp/acl/github.yaml

Then calling the action with { path: "/tmp/acl", destination: "gs://bucket/.acl/", parent: false }

sethvargo commented 1 year ago

Hi @pratheekrebala

The following should work:

uses: google-github-actions/upload-cloud-storage@v1
with:
  path: /tmp/acl.yaml
  destination: bucket/.acl/
  parent: false

This action does not support renaming files before upload, since that would be extremely complex to model for large uploads. If you want more control over the upload, you can use the gsutil or gcloud storage commands directly from the Google Cloud SDK instead.