After upgrade for version 4.0.0 of dependencie azure-pipelines-task, Archive2 does not work anymore

TheJamylle commented 1 year ago

Yesterday after the commit for upgrade, it started to not work anymore, right after this https://github.com/microsoft/azure-pipelines-tasks/commit/e526642ad239018a1227109d12d93a182405a6fc :

Error logs

Found 0 files
/usr/bin/zip -r /home/vsts/work/1/a/6256.zip

zip error: Nothing to do! (/home/vsts/work/1/a/6256.zip)
##[error]Error: Archive creation failed for archive file: /home/vsts/work/1/a/6256.zip 
code: 12 
stdout: 
zip error: Nothing to do! (/home/vsts/work/1/a/6256.zip)

stderr:  
error: undefined;
##[error]Archive creation failed for archive file: /home/vsts/work/1/a/6256.zip 
code: 12 
stdout: 
zip error: Nothing to do! (/home/vsts/work/1/a/6256.zip)

stderr:  
error: undefined;

My yml

- task: ArchiveFiles@2
      displayName: 'Archive files'
      inputs:
        rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
        includeRootFolder: false
        archiveType: 'zip'
        archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
        replaceExistingArchive: true

CaioDS commented 1 year ago

I've same issue.....

Logs:

stderr:  
error: Error: spawnSync /usr/bin/zip ENOBUFS
    at Object.spawnSync (node:internal/child_process:1111:20)
    at Object.spawnSync (node:child_process:814:24)
    at ToolRunner.execSync (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/node_modules/azure-pipelines-task-lib/toolrunner.js:879:23)
    at zipArchive (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:152:33)
    at createArchive (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:278:13)
    at doWork (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:338:9)
    at Object.<anonymous> (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:347:1)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32) {
  errno: -105,
  code: 'ENOBUFS',
  syscall: 'spawnSync /usr/bin/zip',
  path: '/usr/bin/zip',
  spawnargs: [Array]
};
##[error]Archive creation failed for archive file: /home/vsts/work/1/a/3470.zip

Task implementation:

- task: ArchiveFiles@2
  inputs:
    rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
    includeRootFolder: false
    archiveType: 'zip'
    archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
    replaceExistingArchive: true

leni-msft commented 1 year ago

exactly the same issue with @CaioDS

==============================================================================
Task         : Archive files
Description  : Compress files into .7z, .tar.gz, or .zip
Version      : 2.211.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/archive-files
==============================================================================

dcarnelossi commented 1 year ago

I've same issue.....

Logs:

stderr:  
error: Error: spawnSync /usr/bin/zip ENOBUFS
    at Object.spawnSync (node:internal/child_process:1111:20)
    at Object.spawnSync (node:child_process:814:24)
    at ToolRunner.execSync (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/node_modules/azure-pipelines-task-lib/toolrunner.js:879:23)
    at zipArchive (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:152:33)
    at createArchive (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:278:13)
    at doWork (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:338:9)
    at Object.<anonymous> (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:347:1)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32) {
  errno: -105,
  code: 'ENOBUFS',
  syscall: 'spawnSync /usr/bin/zip',
  path: '/usr/bin/zip',
  spawnargs: [Array]
};
##[error]Archive creation failed for archive file: /home/vsts/work/1/a/3470.zip

Task implementation:

- task: ArchiveFiles@2
  inputs:
    rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
    includeRootFolder: false
    archiveType: 'zip'
    archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
    replaceExistingArchive: true

Same issue in all pipelines here.

CaioDS commented 1 year ago

@dcarnelossi @leni-msft I solved this problem by generating the zip file by the powershell core!

My old zip file generation task:

- task: ArchiveFiles@2
  inputs:
    rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
    includeRootFolder: false
    archiveType: 'zip'
    archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
    replaceExistingArchive: true

I replaced it with:

- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: |
      Compress-Archive -Path "$(System.DefaultWorkingDirectory)\*" -DestinationPath "$(Build.ArtifactStagingDirectory)\$(Build.BuildId).zip"
    pwsh: true

adzchappers commented 1 year ago

Experiencing the exact same issue, we have the below code, which suddenly stopped working today between 1149-1317 (GMT)

- task: ArchiveFiles@2
  inputs:
    rootFolderOrFile: '$(Build.ArtifactStagingDirectory)'
    includeRootFolder: false

Adding the archiveType and choosing tar will archive the file into a tar file, however this means I would also need to update our release process. Which I don't want to do as I have hundreds of builds and releases that would need to be updated.

I think this commit might be connected: https://github.com/microsoft/azure-pipelines-tasks/commit/178868ef50485c827548770e207c2d8cf251f62f

AdlerJS commented 1 year ago

Same issue here as well. All builds just stopped working today for ArchiveFiles@2 task.

steps:
- task: ArchiveFiles@2
  displayName: 'Archive Service Files'
  inputs:
    rootFolderOrFile: 'people-service'
    includeRootFolder: false
    archiveFile: '$(Build.ArtifactStagingDirectory)/drop.zip'

I've tried the powershell solution offered by @CaioDS but because our node app is a lerna repo that uses shared local packages the zip creation does not create correctly and fails to deploy. I'll post any updates to this issue via edit if I can solve it

johnnyrwest commented 1 year ago

@dcarnelossi @leni-msft I solved this problem by generating the zip file by the powershell core!

My old zip file generation task:

- task: ArchiveFiles@2
  inputs:
    rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
    includeRootFolder: false
    archiveType: 'zip'
    archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
    replaceExistingArchive: true

I replaced it with:

- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: |
      Compress-Archive -Path "$(System.DefaultWorkingDirectory)\*" -DestinationPath "$(Build.ArtifactStagingDirectory)\$(Build.BuildId).zip"
    pwsh: true

@CaioDS Just noting that our team tried this out with no luck 😕 The pipeline built past the archive task, but the site crashed after releasing. Redeploying an old release for the time being...

skillbuilderzone commented 1 year ago

Exact same issue

stderr:
error: Error: spawnSync /usr/bin/zip ENOBUFS at Object.spawnSync (node:internal/child_process:1111:20) at Object.spawnSync (node:child_process:814:24) at ToolRunner.execSync (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/node_modules/azure-pipelines-task-lib/toolrunner.js:879:23) at zipArchive (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:152:33) at createArchive (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:278:13) at doWork (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:338:9) at Object. (/home/vsts/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:347:1) at Module._compile (node:internal/modules/cjs/loader:1105:14) at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10) at Module.load (node:internal/modules/cjs/loader:981:32) { errno: -105, code: 'ENOBUFS', syscall: 'spawnSync /usr/bin/zip', path: '/usr/bin/zip', spawnargs: [Array] };

Any workaround for now please.

skillbuilderzone commented 1 year ago

@dcarnelossi @leni-msft I solved this problem by generating the zip file by the powershell core!

My old zip file generation task:

- task: ArchiveFiles@2
  inputs:
    rootFolderOrFile: '$(System.DefaultWorkingDirectory)'
    includeRootFolder: false
    archiveType: 'zip'
    archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
    replaceExistingArchive: true

I replaced it with:

- task: PowerShell@2
  inputs:
    targetType: 'inline'
    script: |
      Compress-Archive -Path "$(System.DefaultWorkingDirectory)\*" -DestinationPath "$(Build.ArtifactStagingDirectory)\$(Build.BuildId).zip"
    pwsh: true

This workaround helped us.

mattford commented 1 year ago

I was able to work around this by specifying the pre-upgrade version of the task:

- task: ArchiveFiles@2.206.0

Upgrade appears to be in https://github.com/microsoft/azure-pipelines-tasks/commit/178868ef50485c827548770e207c2d8cf251f62f

adzchappers commented 1 year ago

Another workaround is to use bash, and run a manual zip inline. This seems to avoid the issue around node

- task: Bash@3
  inputs:
    targetType: 'inline'
    script: |
      cd $(Build.ArtifactStagingDirectory) && /usr/bin/zip -r $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip .

denisyakimov07 commented 1 year ago

Same issue

stderr:  
error: Error: spawnSync /usr/bin/zip ENOBUFS
    at Object.spawnSync (node:internal/child_process:1111:20)
    at Object.spawnSync (node:child_process:814:24)
    at ToolRunner.execSync (/Users/runner/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/node_modules/azure-pipelines-task-lib/toolrunner.js:879:23)
    at zipArchive (/Users/runner/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:152:33)
    at createArchive (/Users/runner/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:278:13)
    at doWork (/Users/runner/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:338:9)
    at Object.<anonymous> (/Users/runner/work/_tasks/ArchiveFiles_d8b84976-e99a-4b86-b885-4849694435b0/2.211.0/archivefiles.js:347:1)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32) {
  errno: -55,
  code: 'ENOBUFS',
  syscall: 'spawnSync /usr/bin/zip',
  path: '/usr/bin/zip',
  spawnargs: [Array]
};

And Extract files task has the same issue

error: Error: spawnSync /usr/bin/unzip ENOBUFS
    at Object.spawnSync (node:internal/child_process:1111:20)
    at Object.spawnSync (node:child_process:814:24)
    at ToolRunner.execSync (/Users/runner/work/_tasks/ExtractFiles_5e1e3830-fbfb-11e5-aab1-090c92bc4988/1.211.0/node_modules/azure-pipelines-task-lib/toolrunner.js:879:23)
    at unzipExtract (/Users/runner/work/_tasks/ExtractFiles_5e1e3830-fbfb-11e5-aab1-090c92bc4988/1.211.0/extractfilestask.js:189:35)
    at extractFiles (/Users/runner/work/_tasks/ExtractFiles_5e1e3830-fbfb-11e5-aab1-090c92bc4988/1.211.0/extractfilestask.js:289:17)
    at doWork (/Users/runner/work/_tasks/ExtractFiles_5e1e3830-fbfb-11e5-aab1-090c92bc4988/1.211.0/extractfilestask.js:319:9)
    at Object.<anonymous> (/Users/runner/work/_tasks/ExtractFiles_5e1e3830-fbfb-11e5-aab1-090c92bc4988/1.211.0/extractfilestask.js:328:1)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32) {
  errno: -55,
  code: 'ENOBUFS',
  syscall: 'spawnSync /usr/bin/unzip',
  path: '/usr/bin/unzip',
  spawnargs: [Array]
};

atakane commented 1 year ago

Another workaround is to use bash, and run a manual zip inline. This seems to avoid the issue around node
- task: Bash@3
  inputs:
    targetType: 'inline'
    script: |
      cd $(Build.ArtifactStagingDirectory) && /usr/bin/zip -r $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip .

I believe this one is the best solution as it won't change in time because of a node version update of a component by a team member that causes lost of countless hours around the world.

There is a small fix needed though. The source folder should be $(System.DefaultWorkingDirectory) instead of $(Build.ArtifactStagingDirectory)

    - task: Bash@3
      displayName: 'Archive files with Inline Bash'
      inputs:
        targetType: 'inline'
        script: |
          cd $(System.DefaultWorkingDirectory) && /usr/bin/zip -r $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip .

A more robust version might be using script instead of task

    - script: zip -r --symlinks '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip' .
      displayName: 'Archive files with symlinks'

ryan-bunkr commented 1 year ago

Same issue here.

I tried @CaioDS solution but our code is a node app built from a lerna monorepo and some of the dependencies are symlinked. For whatever reason, the Azure Archive successfully bundles our nested/symlinked dependencies whereas the Compress-Archive script missed all of the symlinked ones and potentially some of the nested deps.

We wanted to implement @mattford solution, but our build pipelines are built with the classic editor instead of custom yaml files. Nevertheless inspired, we finally got things working again by selecting the 1.x task version from the classic editor dropdown for the Azure Archive task.

Classic Editor solution:

So far we aren't seeing any real consequences from downgrading to 1.x but YMMV.

EDIT: we did not try @atakane solution but that looks like it could have handled the symlinks.

Thanks for everyone's workarounds!

atakane commented 1 year ago

Same issue here.

I tried @CaioDS solution but our code is a node app built from a lerna monorepo and some of the dependencies are symlinked. For whatever reason, the Azure Archive successfully bundles our nested/symlinked dependencies whereas the Compress-Archive script missed all of the symlinked ones and potentially some of the nested deps.

We wanted to implement @mattford solution, but our build pipelines are built with the classic editor instead of custom yaml files. Nevertheless inspired, we finally got things working again by selecting the 1.x task version from the classic editor dropdown for the Azure Archive task.

Classic Editor solution:

So far we aren't seeing any real consequences from downgrading to 1.x but YMMV.

Give this one a try as it zips symlinks also

  - script: zip -r --symlinks '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip' .
    displayName: 'Archive files with symlinks'

ryan-bunkr commented 1 year ago

@atakane your solution zips our target directory but is still missing the nested node_module dependencies. I don't know exactly why the Azure Archive step correctly zips them but we gotta just roll with it for now.

Thanks for your solution! It'd probably work if I spent more time figuring out whats going on.

leonsbuddydave commented 1 year ago

I was able to work around this by specifying the pre-upgrade version of the task:
- task: ArchiveFiles@2.206.0
Upgrade appears to be in 178868e

This was the answer for us; works perfectly. Our task after changes:

- task: ArchiveFiles@2.206.0
    inputs: 
      rootFolderOrFile: .
      includeRootFolder: false
      archiveFile: ${{ parameters.outputZipPath }}

This was after a day working on custom zip/Compress-Archive solutions and running into the symlinking issues described above in our NextJS app.

EDIT: To add, thank you @mattford for this solution. I wasn't aware until this that you could refer to tasks by minor version.

atakane commented 1 year ago

@ryan-bunkr We have several other steps to clear caches and install node and packages from scratch. I don't know if it helps you but here is the build stage for our node + nextJS deploy pipeline.

- stage: Build
  displayName: Build stage
  jobs:
  - job: Build
    displayName: Build
    pool:
      vmImage: $(vmImageName)

    steps:
    - task: NodeTool@0
      inputs:
        versionSpec: '16.x'
      displayName: 'Install node.js'

    - script: npm update
      displayName: 'Update packages and install missing ones'

    - task: Cache@2
      displayName: 'Cache node_modules'
      inputs:
        key: node_modules | $(Agent.OS) | package.json
        path: '$(System.DefaultWorkingDirectory)/node_modules'

    - task: Cache@2
      displayName: 'Cache .next/cache'
      inputs:
        key: next | $(Agent.OS) | package.json
        path: '$(System.DefaultWorkingDirectory)/.next/cache'

    - script: npm run build
      displayName: 'Build for prod'

    - script: zip -r --symlinks '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip' .
      displayName: 'Archive files with Bash zip while preserving symlinks'

    - upload: $(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip
      artifact: drop

liester commented 1 year ago

We also noticed this same issue happens in the Extract Files task. Unfortunately, we use the pipeline UI in Azure and there doesn't appear to be a way to downgrade back to a "working" version. Looking at the previous logs it looks like V1.200.0 worked while V1.211.0 does not.

Any ideas on how I can select a specific version (V1.200.0 specifically) in the Extract Files task?

Example of the task selection in azure devops

ryan-bunkr commented 1 year ago

@liester I dont know of a way other than converting the entire pipeline to yaml :/

adzchappers commented 1 year ago

Just ran a build pipeline that I hadn't updated with the bash workaround i mentioned earlier, and this was still ArchiveFiles@2, and it looks as though it connected to 2.206.0, not sure if this was some kind of cache on the build, or perhaps the newer version has been rolled back?

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days

microsoft / azure-pipelines-tasks