actions / upload-artifact

MIT License
3.23k stars 719 forks source link

upload-artifact does not retain artifact permissions #38

Open kcgen opened 4 years ago

kcgen commented 4 years ago

The baseline behavior of the zip utilty on Linux and macOS is to retain permissions.

However, when the upload-artifact action zips a directory, it loses permissions, which subsequently breaks the artifacts for users and downstream tools.

Expected behavior: the permissions applied to assets in prior steps should be retained by the upload-artifact zipper, and should be present in the resulting asset zip file.

kcgen commented 4 years ago

Adding this issue to the download-artifact action, as that's where the zipping is currently performed in the v1 API. https://github.com/actions/download-artifact/issues/14. Will leave this open in both repos open until it's solved (however github wants to manage this in their backend for v2).

nhooyr commented 4 years ago

Can confirm that with unzip -Z on the resulting .zip, there are no executable permissions on files that should have executable perms.

jrozner commented 4 years ago

Is anything being done about this? It's been over 6 moths and this is the recommended way by GitHub to handle artifact upload

adriangb commented 4 years ago

This is still broken as of today, very annoying.

bcardiff commented 4 years ago

I am not proud of it, but actions/cache@v2 keeps the permissions. If the goal is to pass some artifacts between jobs the following is working for me so far.

      - name: Artifact (with permissions)
        uses: actions/cache@v2
        with:
          path: /path/to/store
          key: artifacts-${{ github.run_id }}-${{ github.run_number }}
thetroy commented 4 years ago

A workaround is documented in the readme

https://github.com/actions/upload-artifact#maintaining-file-permissions-and-case-sensitive-files

  - name: 'Tar files'
    run: tar -cvf my_files.tar /path/to/my/directory

  - name: 'Upload Artifact'
    uses: actions/upload-artifact@v2
    with:
      name: my-artifact
      path: my_files.tar  
f0rkz commented 3 years ago

Wow. This is still an issue?

Paebbels commented 3 years ago

@konradpabjan @joshmgross any comments from the authors of this action?

When are you going to fix this BUG?
This makes GitHub workflows in whole a useless tool compared to your competitors offering CI solutions.

lsegal commented 3 years ago

FYI this has probably been mentioned in passing but this completely breaks uploading of binaries as artifacts on macOS, such as .apps etc.

abulka commented 3 years ago

@lsegal agreed - the workaround in my case was to downlad the artifact - uses: actions/download-artifact@v2, add a step to repair any broken permissions with a chmod e.g.

  - run:  chmod +x myapp-macos/myapp.app/Contents/MacOS/myapp

zip up the .app (directory), then uses: softprops/action-gh-release@v1 to upload the zip as a GitHub release.

ghost commented 3 years ago

Uploading as a release actually sounds like a nice idea, as some of us just want to make artifacts directly downloadable by users, so "just chmod +x on download" is not a good solution, because it will be downloaded by the user and not by another workflow, so they have to remember to do it, or I have to make it a nested zip, which is ugly.

Though it could get spammy if a release is posted for each commit rather than just linking it to the workflows, for a project that doesn't really have versioning.

Paebbels commented 3 years ago

Applying chmod manually is no solution. It's easy a job of hundreds to thousands of files.

Also from system architecture is a gigantic faul if the receiving system needs to know how the transmitting system has configured access rights. This breaks encapsulation!

MCOfficer commented 3 years ago

Applying chmod manually is no solution. It's easy a job of hundreds to thousands of files.

Also from system architecture is a gigantic faul if the receiving system needs to know how the transmitting system has configured access rights. This breaks encapsulation!

As mentioned before, the actual workaround for preserving permissions (not restoring them afterwards) is to upload tarballs.

RA80533 commented 3 years ago

@brcrista Any word on this? Would anyone from the team be willing to review and possibly merge a PR fixing this behavior?

MCOfficer commented 3 years ago

@brcrista Any word on this? Would anyone from the team be willing to review and possibly merge a PR fixing this behavior?

https://github.com/actions/upload-artifact/issues/3#issuecomment-598820814

Based on this comment and the fact that the cache action preserves permissions, I'd assume the permissions are actually preserved on the github server; it's only the zipping during the download that discards it.

If that is correct, the code that needs fixing is in fact the download UI, i.e. nothing that's open source.

brcrista commented 3 years ago

I don't think we'll be getting to this anytime soon as we have bigger initiatives in progress. But we have looked at this and here is our understanding of what a fix would take:

So yes, as @MCOfficer pointed out, most of this work is on the service side.

robbert-vdh commented 3 years ago

The permissions issue isn't with the zipping (or if it is, that is easy to fix)

It is. .zip files simply don't store any of the meta data that's used on platforms outside of Windows and DOS. That's why on Linux and other Unix-like platforms people never use .zip files, and they use compressed tarballs instead.

lsegal commented 3 years ago

It is. .zip files simply don't store any of the meta data that's used on platforms outside of Windows and DOS. That's why on Linux and other Unix-like platforms people never use .zip files, and they use compressed tarballs instead.

This is actually the opposite. zip does maintain executable bit and other permissions on Unix platforms. macOS uses zip for its standard compression just fine. It's actually Windows platforms that don't respect exec bits and such, but fortunately Windows doesn't need it: https://unix.stackexchange.com/questions/313656/preserving-permissions-while-zipping

There's nothing fundamentally wrong with using zip files. It is actually default behavior to store/restore permissions, which makes this bug all the more odd that it's somehow not working here. It's almost as if this action is explicitly doing something to break zip's behavior.

robbert-vdh commented 3 years ago

You're right, it does preserve most permissions! Probably shows that I never use .zip files. :grin:

That being said, I'd appreciate it if GitHub Actions would just let me upload tarballs. Right now we end up with these zipped tarballs which no doubt result in some head scratching when users try to download builds from the CI.

melroy89 commented 2 years ago

So is this all on purpose? Or will upload-artifact fix this issue??

ThatXliner commented 2 years ago

Bump. I need this

mxcl commented 2 years ago

I admire GitHubโ€™s desire and ambition to reinvent tar.

Reinventing wheels because โ€œwe can do it betterโ€ is an age-old tradition in software engineering and it would certainly depress me to find out that an $8B company would feel it was beneath them.

I look forward to reporting fresh bugs to the inevitable @github/gh-tar and feeling that wholesome, warm, heady glow of nostalgia when I refer back to the same bugs already fixed 20 years ago by GNU.

refs https://github.com/actions/upload-artifact/issues/38#issuecomment-888585130

jmalins commented 2 years ago

An alternative workaround, if you need to keep the artifact format as a pure ZIP is used here. The approach is to use the getfacl / setfacl CLI tools to backup the permissions into a file that is included in the artifact upload, then restore them from the file on download.

The tools are present in the GHA ubuntu-latest image, so the require setup is minimal.

jupe commented 2 years ago

waiting for fix too.

Joknaa commented 2 years ago

Same for me, I was building an APK using Unity (floatingIslands is the project/APK name). The building was fine, uploaded artifact fine, but didnt have permission to download it to create a release :/

This is the log of the workflow:

Run actions/download-artifact@v3 with: name: FloatingIslands-v1.0.0.33 path: build/Android Starting download for FloatingIslands-v1.0.0.33 Directory structure has been setup for the artifact Total number of files that will be downloaded: 1 node:events:368 throw er; // Unhandled 'error' event ^

Error: EACCES: permission denied, open '/home/runner/work/FloatingIslands/FloatingIslands/build/Android/Android.apk' Emitted 'error' event on WriteStream instance at: at emitErrorNT (node:internal/streams/destroy:157:8) at emitErrorCloseNT (node:internal/streams/destroy:122:3) at processTicksAndRejections (node:internal/process/task_queues:83:21) { errno: -13, code: 'EACCES', syscall: 'open', path: '/home/runner/work/FloatingIslands/FloatingIslands/build/Android/Android.apk' }

klepiz commented 2 years ago

Is this going to be fixed? This bug its still present and it's been almost a year for something super essential as is keep the artifacts permissions

jozefizso commented 1 year ago

We are building macOS app and archiving it. The archive is zipped by the upload artifact actions and the executable will lost +x flag. This makes the app non functional when downloaded from GitHub Actions artifact.

fluffy commented 1 year ago

any progress on this ?

montao commented 1 year ago

We who paid for github actions should get a refund. It's obviously not working.

koliyo commented 11 months ago

Wow, this is a pretty massive problem imo, and really frustrating to see that nothing has happened since 2019 ๐Ÿ˜ฎโ€๐Ÿ’จ

Wonder what the accumulated time spent to fix this locally for users of this action has been since then??

Please fix this ๐Ÿ™

pdmtt commented 8 months ago

This issue still persists as for 2024. Just lost some time trying to figure out why a downloaded artifact's permissions weren't as I expected them to be.

rizface commented 7 months ago

I don't know whether this is a good way or not, whe i trying upload and download artifact which is a golang binary, i set permissions for the binary file on the job that needs execute it

chmod 777 <binary filename>
Vacxe commented 7 months ago

Use tar for zipping archive and upload it as artefact Example: https://github.com/danger/kotlin/blob/master/.github/workflows/publish_release.yml#L60

rmunn commented 6 months ago

Looks like there's a PR that would fix half of this issue: https://github.com/actions/toolkit/pull/1609. (Edit: Nope, see below). That would store file permissions in the .zip files created by upload-artifact; the next step after that would be to fix download-artifact to recreate those permissions, but that can't happen until the permissions actually exist in the .zip file.

But that PR has been sitting there for four months with no review from the GitHub Actions team yet. Hopefully someone from the GHA team will notice that PR and give it some attention soon.

UPDATE: Looks like that PR wouldn't work (the zip.append method wants a mode property, not stats) so I created https://github.com/actions/toolkit/pull/1723 which is basically https://github.com/actions/toolkit/pull/1609 but passing mode into zip.append.

nsmithtt commented 2 months ago

5 years later, still an issue. Couldn't the fix just be to tar before zipping automatically or just enable an option to use tar.gz instead of zip?

rcdailey commented 2 months ago

Here is my workaround: https://gist.github.com/rcdailey/cd3437bb2c63647126aa5740824b2a4f

Basically involves three files, and two custom actions:

.github/
โ””โ”€โ”€ .actions/
    โ”œโ”€โ”€ .download-tar/
    โ”‚   โ”œโ”€โ”€ action.yml
    โ”‚   โ””โ”€โ”€ untar.sh
    โ””โ”€โ”€ .upload-tar/
        โ””โ”€โ”€ action.yml

Examples of how they're used in my workflow jobs:

- name: Upload Artifacts
  uses: ./.github/actions/upload-tar
  with:
    name: ${{ matrix.runtime }}
    path: publish

- name: Download Artifacts
  uses: ./.github/actions/download-tar
  with:
    name: ${{ matrix.runtime }}
    path: publish

They're very simplistic (mainly because this is all I need for my particular project) but I'm happy with them and they do their job. Hopefully these custom actions will help others.