[JENKINS-64818] Job artifacts zip archive format change breaks external tool

[timja]

Since Jenkins 2.263.2 the format of the artifact Zip archive changed in a way that breaks the Azure DevOps plugin we are using to download Jenkins job artifacts after a job has finished. The archives downloaded from Jenkins 2.263.1 and 2.263.2+ contain exactly the same files (as expected), but differ when using binary comparison. This was tested by downloading artifacts from the same build in version 2.263.1 and 2.263.2/2.263.3. I attached one archive downloaded from 2.263.1 and one archive downloaded from 2.263.3. 

Since there is no easy workaround we had to roll back to 2.263.1.

---

Originally reported by bb, imported from: Job artifacts zip archive format change breaks external tool

• status: Open
• priority: Major
• resolution: Unresolved
• imported: 2022/01/10

[timja]

markewaite:

It seems that you're relying on a binary comparison of the zip file itself. Isn't that much more brittle than is required?

The purpose of the zip file is to provide all the files and directories from the source location in the workspace. Relying on the precise ordering of files in the zip archive or precise compression levels or other internal details of the zip file format seems more precise than the zip archive of files in the workspace was ever intended to be. Since the changes in Jenkins 2.263.2 resolved one or more security issues with that change, I doubt that the maintainers will be willing to work to provide that level of compatibility.

[timja]

bb:

It surely is way too brittle, but the plugin we are using was not written by me. I also filed a bugreport for that plugin in the Azure DevOps issue tracker already. I just thought it would be a good idea to at least make that hidden change public here.

I don´t even know if that specific change is breaking the plugin in question or if there is more to it. But it is the only behavioural change between Jenkins 2.263.1 and 2.263.3 I could find.

[timja]

ianw:

I wonder if this isrelated - Backup plungin in jenkins is not able to restore the backupzip in linux

[timja]

john_fo:

This was reported to Azure Devops and analysed their side. See https://github.com/microsoft/azure-pipelines-tasks/issues/14320. Their analysis is that the problem is in the meta data of the zip file generated by Jenkins. I don't know enough to comment but seems we are currently caught in the middle on this. Only known workaround to date is to stick to old Jenkins builds.

markewaite do you happen to know who to poke on this?

[timja]

JIRAUSER132320:

I got here from https://github.com/microsoft/azure-pipelines-tasks/issues/14320 We are using Jenkins docker image 2.277.4. Builds are completed successfully in Jenkins server and errors during artifacts download in Azure Devops.

 
/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/artifact-engine/Engine/artifactEngine.js:157
throw err;
^
ExtractionFailed Error: Unsupported file type "undefined"
at DecompressZip.extractFile (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:309:11)
at /workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:242:28
at Array.forEach (native)
at DecompressZip.extractFiles (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:241:11)
at /workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/decompress-zip/lib/decompress-zip.js:121:21
at _fulfilled (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:854:54)
at self.promiseDispatch.done (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:883:30)
at Promise.promise.promiseDispatch (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:816:13)
at /workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:624:44
at runSingle (/workspace/_tasks/JenkinsDownloadArtifacts_86c37a92-59a7-444b-93c7-220fcf91e29c/1.175.0/node_modules/q/q.js:137:13)
##[warning]Can\'t find loc string for key: ExtractionFailed
 

[timja]

JIRAUSER132911:

Something appears to have changed with the meta-data in Jenkins generated zip files between these versions. Both the versioning and how external attributes are written.

This caused an issue with the Azure Devops task dependency decompress-zip 0.3.0 but it appears it is fixed in decompress-zip 0.3.3.

I confirmed 0.3.0 fails with the given error message when attempting to extract the new  zip files whereas 0.3.3 successfully extracts them. Both with the attached files, and with files from Jenkins 2.289.1

https://github.com/bower/decompress-zip/issues/71

It appears decompress-zip deems the new zip files to be from UNIX instead of DOS, assuming the "External attributes" regarding file-type to be in a particular format, and the way it expects this format to describe file-type doesn't match what's in the Jenkins generated zip file, which causes it to fail.

I have let the Azure Devops team know, and hopefully this issue should be closed by them updating their dependencies in the near future.