search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.49k stars 2.61k forks source link

DownloadPipelineArtifact permissions lost on ubuntu #12002

Closed Justin-DynamicD closed 8 months ago

Justin-DynamicD commented 4 years ago

Type: Bug Enter Task Name: DownloadPipelineArtifact

Issue Description

When using PublishPipelineArtifact with DownloadPipelineArtifact on ubuntu-latest (assuming other distros as well), the artifacts published then downloaded will not maintain execution rights.

An example of this would be using the -outfile option in Terraform, which will bundle downloaded plugins into the generated artifact. If the plugin files do not have execution permissions set once extracted to the target deployment agent, the subsequent Terraform steps will fail due to permission issues.

As a quick workaround I run a bash script after the DownloadPipelineArtifact step to chmod +x the plug-ins, but it would be preferable to maintain the file permissions using the task itself.

t3mi commented 4 years ago

The other case - release shell scripts with proper defined execute permission in git which go to artifact as simple files. And during release that permission is lost and bash task prints at least a warning about that. That's definitely a bug.

herenhuang commented 4 years ago

Just to give an update here from the Artifacts team. It does seem like a bug, so we'll track this like one and update here when possible.

Lectem commented 4 years ago

Same issue here, and I have a lot of executables bundled in the artifact. Using chmod +x on all of them is not very practical.

It does seem like a bug

I think this should probably be documented in the meantime.

Justin-DynamicD commented 4 years ago

@herenhuang thank you for the update. I looks like it was changed from a bug to an enhancement in Jan, so if it is agreed to be a bug, we may need to flip that back.

ElvenSpellmaker commented 4 years ago

Just ran into this issue too, it's very surprising to not keep permissions, running a chmod is a bit of a hack tbh.

tusharb86 commented 4 years ago

@herenhuang Hi there - I'm trying to understand why this is considered an enhancement instead of a bug? Is it required that consumers of artifacts keep a list of executable files and mark them as executable (chmod +x) after the artifact is downloaded? This certainly feels like a bug in the implementation of artifacts. Many teams are having to put in their own hack/workaround to deal with this issue. Is there any plan to work on improving this in the near future?

Lectem commented 3 years ago

Note that github actions followed the same path and has the same issue. to mitigate this they document (only on the upload task, not download, which is not that easy to find) that one can simply tar.gzip the directory before (but then you have to do filtering of files yourself).

I think an Ok(ish) solution would be to have the task use tar.gz for storage instead of Zip based on a parameter of the task At least it would be easily discovered and fixable.

github-actions[bot] commented 3 years ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

ElvenSpellmaker commented 3 years ago

How do we remove the stale tag?

ElvenSpellmaker commented 3 years ago

It's ridiculous Microsoft can just close stuff with a bot without any interaction from people.

Justin-DynamicD commented 3 years ago

Original owner back at it: I have clever work arounds but but I cant escape the feeling that this just ... isn't how it should be.

Is this at least being road-mapped?

github-actions[bot] commented 2 years ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

tusharb86 commented 2 years ago

@fadnavistanmay @carl-tanner Any update on this? I just saw the stale tag and wanted to ensure this doesn't get closed due to inactivity.

Justin-DynamicD commented 2 years ago

Original Poster again: this still exists. It's also persisted into Github Actions. It also is still falsely categorized as an Enhancement instead of a bug.

Please acknowledge.

AlastairHay commented 2 years ago

Suffering from this (two years later) when uploading plugins in the .terraform directory as per https://learn.hashicorp.com/tutorials/terraform/automate-terraform#plan-and-apply-on-different-machines

Permissions are being mangled by the publish then download. Workaround that I have not yet tried is to tar the directory before publishing.

Agree that this is a bug not an enhancement.

github-actions[bot] commented 2 years ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

craigktreasure commented 2 years ago

Still a problem.

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

craigktreasure commented 1 year ago

Still interested in this.

SoftwareApe commented 1 year ago

How can this obvious bug still be there after 3 years?

ElvenSpellmaker commented 1 year ago

How can this obvious bug still be there after 3 years?

Because Microsoft.

Oh and Azure DevOps Pipelines is basically being replaced with Github Actions so a lot of their efforts are on that instead now. The host images for AzDo are Github Actions images now with the AzDo runtime slapped on top. (Their docs for AzDo lead you to the GHA Github page).

AchimTuran commented 1 year ago

So far I know per default the artifacts are stored in zip files, which does not support file permissions. But we can use those tasks to work with tar files nowadays. https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/archive-files-v2?view=azure-pipelines https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/extract-files-v1?view=azure-pipelines

SoftwareApe commented 1 year ago

@AchimTuran that's the same workaround we use right now, although it would be nice if publish and download "just worked". Maybe using .tar.gz by default on Linux and MacOs agents would be enough.

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

SoftwareApe commented 1 year ago

Not stale

github-actions[bot] commented 8 months ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

samuel-laplace commented 4 months ago

Any update here?