Open c-eliasson opened 4 years ago
Starting: Cache npm Task : Cache Description : Cache files between runs Version : 2.0.1 Author : Microsoft Corporation Help : https://aka.ms/pipeline-caching-docs Resolving key:
npm|"Windows_NT"|2DZ3WcfStD108j8qipq/0v97TF5eyy6fpvtPYfpDrZM=
Information, There is a cache miss.
tar: Cannot change to directory E:\Agents\3\283\s\node_modules: No such file or directory
tar: Error is not recoverable: exiting now
Information, ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session XXXXXXXX282-485c-a6f4-b92a59ed4955
Finishing: Cache npm
2020-06-09T21:30:32.6150811Z ##[section]Starting: Cache yarn packages
2020-06-09T21:30:32.6163777Z ==============================================================================
2020-06-09T21:30:32.6164193Z Task : Cache
2020-06-09T21:30:32.6164523Z Description : Cache files between runs
2020-06-09T21:30:32.6164828Z Version : 2.0.1
2020-06-09T21:30:32.6165145Z Author : Microsoft Corporation
2020-06-09T21:30:32.6165519Z Help : https://aka.ms/pipeline-caching-docs
2020-06-09T21:30:32.6165941Z ==============================================================================
2020-06-09T21:30:32.9975064Z Resolving key:
2020-06-09T21:30:33.0154988Z - yarn [string]
2020-06-09T21:30:33.0162685Z - "Linux" [string]
2020-06-09T21:30:33.0743802Z - yarn.lock [file] --> EA85735E476B7CE9FF55138A52CE89B7299F92856B621075773F035440C87D93
2020-06-09T21:30:33.0785039Z Resolved to: yarn|"Linux"|QlvZU5gmusF+bBuImFOlW68Qf4aNmdJ1pmhH2LChb5o=
2020-06-09T21:30:33.6622757Z Information, ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session c6edd63d-1238-44dd-a471-3439ac2f402e
2020-06-09T21:30:33.6861181Z Information, Getting a pipeline cache artifact with one of the following fingerprints:
2020-06-09T21:30:33.6862267Z Information, Fingerprint: `yarn|"Linux"|QlvZU5gmusF+bBuImFOlW68Qf4aNmdJ1pmhH2LChb5o=`
2020-06-09T21:30:34.2316763Z Information, There is a cache miss.
2020-06-09T21:30:34.2834153Z tar: /home/vsts/work/1/.yarn: Cannot open: No such file or directory
2020-06-09T21:30:34.2834821Z tar: Error is not recoverable: exiting now
2020-06-09T21:30:34.5635719Z Information, ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session c6edd63d-1238-44dd-a471-3439ac2f402e
2020-06-09T21:30:34.5689658Z ##[error]Process returned non-zero exit code: 2
2020-06-09T21:30:34.5844564Z ##[section]Finishing: Cache yarn packages```
Same issue here with yarn - it is in some autogenerated task after my WAR deploy is done - the task is called "Post-job: Cache":
tar: /blah/blahgent/_work/1/s//node_modules, !/node_modules/**/node_modules: Cannot open: No such file or directory tar: Error is not recoverable: exiting now Information, ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session 18de2057-e96c-4ff9-9091-595b9595f5cf
Finishing: Cache
Any clue how to solve?
Came in here to +1 to this, below is my config:
variables:
- name: yarnCacheFolder
value: $(Pipeline.Workspace)/.yarn
- task: Cache@2
displayName: Cache Yarn Packages
inputs:
key: yarn | $(Agent.OS) | yarn.lock
restoreKeys: |
yarn | $(Agent.OS)
yarn
path: $(yarnCacheFolder)
Post-job: Cache Yarn Packages
will throw an error
/home/vsts/work/1/.yarn: Cannot open: No such file or directory
For me it works, but I have set the directory directly because I found no suitable pipeline variable:
- task: Cache@2
inputs:
key: 'yarn | "$(Agent.OS)" | Frontend/.../yarn.lock'
restoreKeys: yarn | "$(Agent.OS)"
path: /home/vsts/.cache/yarn
displayName: Cache Yarn packages
I'm using a hosted agent (ubuntu-18.04); maybe this works for you as well.
One can print the relevant directory for yarn with yarn cache dir
@jfheins thanks for the hint.
Current configuration works for me in cloud-hosting:
pool:
vmImage: 'ubuntu-latest'
variables:
- name: YARN_CONFIG_CACHE
value: /home/vsts/.cache/yarn/v6
...
- task: Cache@2
displayName: Cache node modules
inputs:
key: 'yarn | "$(Agent.OS)" | yarn.lock'
restoreKeys: |
yarn | "$(Agent.OS)"
path: $(YARN_CONFIG_CACHE)
This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days
Bumping because this is still happening and I don't think it should be closed
I was running into this exact issue and fixed it by adding YARN_CACHE_FOLDER to the pipeline variables.
I had the Cache task as part of a task group and in the pipeline that uses the task group, there is a required field that asks for the value of YARN_CACHE_FOLDER
which is used as 'path' for the Cache task. Filled in the field with $(Pipeline.Workspace)/.yarn
but the post-job kept failing with same error as above.
Once I also defined YARN_CACHE_FOLDER=$(Pipeline.Workspace)/.yarn
in the pipeline variables, the post-job stopped failing.
Bump - this actually just started happening on a build that hasn't changed since December.
Same here for Maven:
variables:
IS_MASTER_OR_DEVELOP: $[in(variables['Build.SourceBranch'], 'refs/heads/master', 'refs/heads/develop')]
MAVEN_CACHE_FOLDER: $(Pipeline.Workspace)/.m2/repository
MAVEN_OPTS: '-Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)'
pool:
vmImage: 'ubuntu-latest'
steps:
- task: Cache@2
displayName: "Cache Maven local repo"
inputs:
key: 'maven | "$(Agent.OS)" | **/pom.xml'
restoreKeys: |
maven | "$(Agent.OS)"
maven
path: $(MAVEN_CACHE_FOLDER)
- task: Maven@3
displayName: "Publish artifacts"
condition: eq(variables.IS_MASTER_OR_DEVELOP, true)
inputs:
mavenPomFile: './pom.xml'
mavenOptions: '-Xmx3072m $(MAVEN_OPTS)'
mavenAuthenticateFeed: true
goals: 'deploy'
Same as @loxy
I'm not sure why the pipeline I created started throwing the /home/vsts/work/1/.yarn: Cannot open: No such file or directory
error as I use the exact same setup other places, with $(Pipeline.Workspace)/.yarn
. I tried using different vmImages with no luck. Here's how I solved it for the ones interested.
- bash: echo "##vso[task.setvariable variable=yarnCacheDir;]$(yarn cache dir)"
displayName: Set yarn cache dir variable
- task: Cache@2
inputs:
key: '"yarn" | "$(Agent.OS)" | yarn.lock'
restoreKeys: |
yarn | "$(Agent.OS)"
path: $(yarnCacheDir)
displayName: Cache Yarn packages
Same here for Maven:
variables: IS_MASTER_OR_DEVELOP: $[in(variables['Build.SourceBranch'], 'refs/heads/master', 'refs/heads/develop')] MAVEN_CACHE_FOLDER: $(Pipeline.Workspace)/.m2/repository MAVEN_OPTS: '-Dmaven.repo.local=$(MAVEN_CACHE_FOLDER)' pool: vmImage: 'ubuntu-latest' steps: - task: Cache@2 displayName: "Cache Maven local repo" inputs: key: 'maven | "$(Agent.OS)" | **/pom.xml' restoreKeys: | maven | "$(Agent.OS)" maven path: $(MAVEN_CACHE_FOLDER) - task: Maven@3 displayName: "Publish artifacts" condition: eq(variables.IS_MASTER_OR_DEVELOP, true) inputs: mavenPomFile: './pom.xml' mavenOptions: '-Xmx3072m $(MAVEN_OPTS)' mavenAuthenticateFeed: true goals: 'deploy'
@loxy Any Suggestion ? I'm also stuck here .
@pradpant No, didn't found a solution
bump!
This was working in a single stage pipeline, once I added another stage that causes this to get called twice, I now get the error with the cache post-job Local agent running Win10 pro
Starting: Cache Yarn packages
==============================================================================
Task : Cache
Description : Cache files between runs
Version : 2.0.1
Author : Microsoft Corporation
Help : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key:
- yarn [string]
- "Windows_NT" [string]
- src/MyProject.Web.Host/yarn.lock [file] --> A51A47DAA3D812F7DBE1028017AC2CA60169EA036A0C8BDD2DE0B6DADB8B21FE
Resolved to: yarn|"Windows_NT"|+Z30KjYR/75Haaf6MF5eU1mX/TrlZ7I5cG6FuT0QJfE=
ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session 7d701a80-cde5-40c0-abed-7f66814cc117
Getting a pipeline cache artifact with one of the following fingerprints:
Fingerprint: `yarn|"Windows_NT"|+Z30KjYR/75Haaf6MF5eU1mX/TrlZ7I5cG6FuT0QJfE=`
There is a cache miss.
tar: could not chdir to 'C:\agent\_work\5\.yarn'
ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session 7d701a80-cde5-40c0-abed-7f66814cc117
##[error]Process returned non-zero exit code: 1
Finishing: Cache Yarn packages
Same here. Seems the cache plugin is broken. Found another solution here: https://xebia.com/blog/caching-your-node-modules-in-azure-devops/
This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days
/remove-stale
I'm also running into this issue, also with two consecutive @Cache2 tasks (one for $(Pipeline.Workspace)/.npm and one for $(Pipeline.Workspace)/.cache/Cypress). Also tried with CacheBeta@1, same issue. The first post-job executed is Cache Cypress (not npm which is the first cache task), which fails with:
There is a cache miss.
tar: /agent/_work/1/.cache/Cypress: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session ***-**-**-**-***
##[error]Process returned non-zero exit code: 2
Finishing: Cache Cypress binary
I think caching will only work on a self hosted agent?
I think caching will only work on a self hosted agent?
No, this task is especially good for the Azure hosted agents. In a self hosted scenario, most of the caching works without this task. (For example: yarn has a machine wide global cache that persists in a self hosted agent)
The tricky thing is to get the cache directory right. For example, an Azure agent mighthave the yarn cache in "/home/vsts/.cache/yarn" while a self hosted might need "/home/AzDevOps/.cache/yarn"
I'd recommend to include a step that logs out the cache directories for easier troubleshooting:
- script: |
node -v
npm -v
yarn -v
yarn cache dir
npm config get cache
displayName: "Log versions & paths"
So, the issue seems to be that $(Pipeline.Workspace)
does not return the home directory and hence the step can't find the build folders that are typically in the home dir. Take the output from the command @jfheins posted and add that to the variable directly.
# example for .npm
# replace this
variables:
CACHE_FOLDER: $(Pipeline.Workspace)/.npm
# with this
variables:
CACHE_FOLDER: /home/vsts/.npm
Here's the result of the successful run after making the change
I have the same error, but with pnpm. This is the yaml from the docs:
variables:
pnpm_config_cache: $(Pipeline.Workspace)/.pnpm-store
steps:
- task: Cache@2
inputs:
key: 'pnpm | "$(Agent.OS)" | pnpm-lock.yaml'
path: $(pnpm_config_cache)
displayName: Cache pnpm
- script: |
corepack enable
corepack prepare pnpm@latest-8 --activate
pnpm config set store-dir $(pnpm_config_cache)
displayName: "Setup pnpm"
- script: |
pnpm install
pnpm run build
displayName: "pnpm install and build"
@vixero-dev Can you share your the log from the task failure?
A lot of these issues look like folks either aren't:
@johnterickson you are correct. The variable wasn't being set.
After changing this part:
- script: |
corepack enable
corepack prepare pnpm@latest-8 --activate
pnpm config set store-dir $(pnpm_config_cache)
displayName: "Setup pnpm"
to this:
- script: |
corepack enable
corepack prepare pnpm@latest-8 --activate
displayName: "setup pnpm"
- script: pnpm config set store-dir $(pnpm_config_cache)
displayName: "setup pnpm config"
I stopped experiencing this issue.
I had some similar issues that ended up being due to agents getting different versions of yarn. I added
- bash: |
corepack enable
echo "Requesting Yarn version $(YARN_VERSION)"
yarn set version $(YARN_VERSION)
echo "Running Yarn $(yarn --version)"
along with configuring the YARN_VERSION
variable to 3.6.4 in my case. I found out some agents would give me 1.22 sometimes in some build steps and 3.6 in others. This helped me diagnose this.
Also experience this issue when following Microsoft's documentation. It is an issue with the paths. This blog posts is a great dive into how to fix this and the Cache task.
Try adding an s
between $(Pipeline.Workspace)
and .yarn
, so variable goes from:
YARN_CACHE_FOLDER: $(Pipeline.Workspace)/.yarn
To:
YARN_CACHE_FOLDER: $(Pipeline.Workspace)/s/.yarn
This is because when using checkout: self
, it checks out the repository to $(Pipeline.Workspace)/s
, as is explained here. Unless your using magic, your .yarn folder should be inside the repository.
@nickshanks347 's comment is really helpful. In deed, the same also happening to other tool such as pip
I tried
variables:
NPM_CACHE_FOLDER: $(Pipeline.Workspace)/s/.npm
# ...
steps:
- task: Cache@2
displayName: Cache NPM packages
inputs:
key: '"npm" | "$(Agent.OS)" | package-lock.json'
restoreKeys: |
"npm" | "$(Agent.OS)"
"npm"
path: $(NPM_CACHE_FOLDER)
cacheHitVar: NPM_CACHE_RESTORED
But I get
Post-job: Cache NPM packages
[View raw log](https://dev.azure.com/landshypotek/980d789b-7245-4c89-a4a0-320898842fa4/_apis/build/builds/9637/logs/19)
Starting: Cache NPM packages
==============================================================================
Task : Cache
Description : Cache files between runs
Version : 2.198.0
Author : Microsoft Corporation
Help : https://aka.ms/pipeline-caching-docs
==============================================================================
Resolving key:
- "npm" [string]
- "Linux" [string]
- package-lock.json [file] --> 0C9C69F14CFB03BABFC87109EECCB2B787FBAD0ECFC7E1F9F823A953E0A5303E
Resolved to: "npm"|"Linux"|NyzAW704Sn4J2Y1PCdbgFC5xaendrqBTYW+d+xSeAhY=
Using default max parallelism.
Using default max parallelism.
Max dedup parallelism: 192
DomainId: 0
ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session ab82fec9-6c06-4dc7-9d91-1f0827ba36b5
Hashtype: Dedup64K
Getting a pipeline cache artifact with one of the following fingerprints:
Fingerprint: `"npm"|"Linux"|NyzAW704Sn4J2Y1PCdbgFC5xaendrqBTYW+d+xSeAhY=`
There is a cache miss.
tar: /home/vsts/work/1/s/.npm: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session ab82fec9-6c06-4dc7-9d91-1f0827ba36b5
##[error]Process returned non-zero exit code: 2
Finishing: Cache NPM packages
I would expect the post-job to create the folder if it doesn't exist.
I would expect the post-job to create the folder if it doesn't exist.
The output is telling you that the directory with the stuff you want to cache does not exist. (i.e. there is nothing to cache)
Have you tried to run npm --global cache verify
in the pipeline to have npm output the cache directory?
Hi, We're having the same issue, but with NuGet packages.
2024-10-07T12:06:28.2977763Z ##[section]Starting: NuGet Cache
2024-10-07T12:06:28.2990900Z ==============================================================================
2024-10-07T12:06:28.2991333Z Task : Cache
2024-10-07T12:06:28.2991508Z Description : Cache files between runs
2024-10-07T12:06:28.2991716Z Version : 2.198.0
2024-10-07T12:06:28.2991911Z Author : Microsoft Corporation
2024-10-07T12:06:28.2992096Z Help : https://aka.ms/pipeline-caching-docs
2024-10-07T12:06:28.2992514Z ==============================================================================
2024-10-07T12:06:28.7416432Z Resolving key:
2024-10-07T12:06:28.7601309Z - nuget [string]
2024-10-07T12:06:28.7646429Z - "Windows_NT" [string]
2024-10-07T12:06:29.3223086Z - **/packages.lock.json,!**/bin/**,!**/obj/** [file pattern; matches: 1]
2024-10-07T12:06:29.3250490Z - s\Exposed-API-v2\packages.lock.json --> EA02F2C348E6430278923F99E7BD2C6199A615EA44D8C5C04AA1ADAB8EDE1223
2024-10-07T12:06:29.3275286Z Resolved to: nuget|"Windows_NT"|14ENw8o70BCYT0eXNadg8S+3a/Qg6UX21F0XdjBe9oI=
2024-10-07T12:06:29.3902883Z Using default max parallelism.
2024-10-07T12:06:30.9012730Z tar: could not chdir to 'D:\a\1\.nuget\packages'
2024-10-07T12:06:30.9014291Z
2024-10-07T12:06:30.9015271Z Using default max parallelism.
2024-10-07T12:06:30.9016140Z Max dedup parallelism: 192
2024-10-07T12:06:30.9016911Z DomainId: 0
2024-10-07T12:06:30.9018238Z ApplicationInsightsTelemetrySender will correlate events with X-TFS-Session d190de86-e08b-403a-a797-9e5695b45cf8
2024-10-07T12:06:30.9019259Z Hashtype: Dedup64K
2024-10-07T12:06:30.9020154Z Getting a pipeline cache artifact with one of the following fingerprints:
2024-10-07T12:06:30.9021249Z Fingerprint: `nuget|"Windows_NT"|14ENw8o70BCYT0eXNadg8S+3a/Qg6UX21F0XdjBe9oI=`
2024-10-07T12:06:30.9022151Z There is a cache miss.
2024-10-07T12:06:30.9454673Z ApplicationInsightsTelemetrySender correlated 1 events with X-TFS-Session d190de86-e08b-403a-a797-9e5695b45cf8
2024-10-07T12:06:30.9509135Z ##[error]Process returned non-zero exit code: 1
2024-10-07T12:06:30.9523659Z ##[section]Finishing: NuGet Cache
Can we have an update on this ? By the way, I agree with @Evanion when he say:
I would expect the post-job to create the folder if it doesn't exist.
With nuget, the process to get it working is the same:
dotnet nuget locals all --list
within the pipeline)Optional: Check what pipeline variables exist and maybe you can replace part of the absolute path with a variable
Required Information
Entering this information will route you directly to the right team and expedite traction.
Question, Bug, or Feature?
Type: Bug
Enter Task Name: CacheV2 (https://github.com/microsoft/azure-pipelines-tasks/tree/master/Tasks/CacheV2)
Environment
Server - Azure Pipelines
Agent - Hosted:
Issue Description
Using the CacheV2 task to cache Yarn packages, we're getting an error in the post-job step of the cache task.
The task is defined exactly as in the docs.
We're using the
vs2017-win2016
image.Error logs