Open ehmicky opened 4 years ago
Thanks for looking at this!
A few notes I forgot to mention yesterday:
YARN_GLOBAL_FOLDER
is the folder where everything global to yarn 2 is stored. For now I think it is only a cache
folder so not really a problem.YARN_ENABLE_PROGRESS_BARS=false
) even though it is already disabled on CI env--non-interactive
option should not cause any problems as yarn install
should never ask for anything$YARN_GLOBAL_FOLDER/cache
might be enough, I'd like to note that some install could have a "local" cache folder (aka Offline mirrors in v1) that is or isn't committed in the repo. It is used mainly for PnP installs (the default in v2) and caching this folder (when not committed in vcs) instead of the global one should be more efficient. node_modules installs should use enableGlobalCache: true
so it shouldn't matter much for those.Also, how could we detect the usage of Yarn v2? ~Another env var?~
Actually the easiest and more reliable way would be too run
yarn --version
and parse result to check version
Thanks a lot for this information, this is very helpful! :heart:
I am using yarn v2 on xstyled project, it works fine but it looks like caching is not really effective. See this build: https://app.netlify.com/sites/xstyled/deploys/60584152ce7ae70008522e1f
I saw that I can configure a custom cache folder using environment variables but I think dependencies caching should work out of the box for common package managers like yarn.
Hi @gregberge and the Netlify team. We were considering trying out Yarn 2, which was released at the beginning of 2020, but are concerned about how little support Netlify seems to have for it.
Wanted to draw your attention to the fact that there's been a few posts to your forums over the last year without any actual movement. One of them claims that adding an extra path configuration file to the repo makes it at least buildable with Yarn 2. https://answers.netlify.com/t/using-the-new-yarn-release-2-0-0-berry/8270
But the issue notes above indicate that there's v1-specific yarn install
code baked into Netlify, so I sort of doubt those workarounds do all that much. (I don't even know if you're supposed to run yarn install
in v2, since the idea with it is that deps are pre-bundled in the repo.)
All of these things make our team wary of even trying it out while using Netlify, although we'd like to -- Yarn 2 is a major version of one of the most popular package managers after all. Any progress to report here? Thanks!
@mosesoak I agree with your general points just wanted to add something to the following:
One of them claims that adding an extra path configuration file to the repo makes it at least buildable with Yarn 2.
I was able to verify that this works for one of my projects.
I don't even know if you're supposed to run yarn install in v2, since the idea with it is that deps are pre-bundled in the repo.
You're probably referring to zero installs here and that is something that is completely optional (see more here). Netlify should probably verify if a project is using zero installs and then don't execute yarn install
.
Any progress on this? It seems that build plugins installed via package.json
also break in Yarn v2/3 (with PnP): https://app.netlify.com/sites/oddleventy/deploys/614a0a6b6bee0100bad715d3
Another confused user here π₯
I am switching a Yarn 3 project to Netlify, but canβt get cache working no matter what I try. The most rational move I could come up was setting these variables in Netlify UI and then trying to run the deploy twice:
YARN_CACHE_FOLDER=/opt/buildhome/cache/.yarn_cache
YARN_ENABLE_GLOBAL_CACHE=true
Regardless of what I do, I keep getting:
10:56:02 PM: β€ YN0000: β Fetch step
10:58:21 PM: β€ YN0013: β 14 packages were already cached, 730 had to be fetched
10:58:21 PM: β€ YN0000: β Completed in 2m 19s
which means more than two minutes are wasted on every build.
I am using Next and my build setup on Netlify is pretty basic:
Base directory | Not set |
Build command | next build |
Publish directory | .next |
Deploy log visibility | Logs are public |
Builds | Active |
# netlify.toml
[build]
publish = ".next"
[[plugins]]
package = "@netlify/plugin-nextjs"
# .yarnrc.yml
nodeLinker: node-modules
yarnPath: .yarn/releases/yarn-3.1.1.cjs
I have even tried setting YARN_CACHE_FOLDER=.next/cache/yarn
in Netlify UI. Despite that Next keeps its own cache, Yarn cache is not becoming a part of it, as if the environment variable is not there.
The oddest thing is this:
11:47:01 PM: Starting to download cache of 289.4MB π π π π π
11:47:03 PM: Finished downloading cache in 1.838382038s
11:47:03 PM: Starting to extract cache
11:47:12 PM: Finished extracting cache in 9.146324277s
11:47:12 PM: Finished fetching cache in 11.039018404s
11:47:12 PM: Starting to prepare the repo for build
Something is being cached somewhere and itβs clearly bigger than Next.js cache (β50MB).
I am giving up after wasting the whole evening π Does anyone have any working solution for Yarn 3? If you do, please share π
I am giving up after wasting the whole evening π Does anyone have any working solution for Yarn 3? If you do, please share π
Hi @kachkaev ,
We don't use Next, we use React App with Create React App (4.0.3) and the yarn cache is working (but it wasn't easy to find the "rigth" setup)
Our netlify.toml:
[build.environment]
NODE_VERSION = "16"
YARN_VERSION = "3.1.1"
YARN_CACHE_FOLDER = "/opt/buildhome/.yarn_cache"
We have .yarnrc and .yarnrc.yml with the same content: (The .yarnrc the official yarn config file and we need it for local development and netlify uses the .yarnrc.yml file)
nodeLinker: node-modules
plugins:
- path: .yarn/plugins/@yarnpkg/plugin-interactive-tools.cjs
spec: "@yarnpkg/plugin-interactive-tools"
yarnPath: .yarn/releases/yarn-3.1.1.cjs
Our netlify build log:
12:49:01 PM: Build ready to start
12:49:03 PM: build-image version: 122b31996ccaffd45d820a452d6227f8312110cc (focal)
12:49:03 PM: build-image tag: v4.5.3
12:49:03 PM: buildbot version: 44e4d661fad7f5ef0b9bffed6dfdedaa2c1ad008
12:49:03 PM: Fetching cached dependencies
12:49:03 PM: Starting to download cache of 730.3MB
12:49:08 PM: Finished downloading cache in 4.930520996s
12:49:08 PM: Starting to extract cache
12:49:31 PM: Finished extracting cache in 22.562380332s
12:49:31 PM: Finished fetching cache in 27.65318033s
...
12:49:40 PM: Started restoring cached yarn cache
12:49:41 PM: Finished restoring cached yarn cache
12:49:42 PM: No yarn workspaces detected
12:49:42 PM: Started restoring cached node modules
12:49:42 PM: Finished restoring cached node modules
12:49:43 PM: Installing NPM modules using Yarn version 3.1.1
12:49:44 PM: β€ YN0050: The cache-folder option has been deprecated; use rc settings instead
...
12:49:45 PM: β€ YN0000: β Fetch step
12:49:46 PM: β€ YN0013: β 1780 packages were already cached
12:49:46 PM: β€ YN0000: β Completed in 0s 957ms
We are getting this warning, so I don't know how long it will work.
12:49:44 PM: β€ YN0050: The cache-folder option has been deprecated; use rc settings instead
I hope this will help you!
Ps.: I checked the Netlify build-image repo to find out how the netlify cache works and it helped a lot.
Thanks a lot for your reply @ZotyoNegentropics β your solution worked! All I had to do in my case was adding these lines to netlify.toml
:
[build]
publish = ".next"
+ ## Yarn 3 cache does not work out of the box as of Jan 2022. Context:
+ ## https://github.com/netlify/build/issues/1535#issuecomment-1021947989
+ [build.environment]
+ YARN_CACHE_FOLDER = "/opt/buildhome/.yarn_cache"
+ YARN_VERSION = "3.1.1"
+
[[plugins]]
package = "@netlify/plugin-nextjs"
Here is the result:
9:47:51 AM: β€ YN0000: β Fetch step
9:47:52 AM: β€ YN0013: β 740 packages were already cached
9:47:52 AM: β€ YN0000: β Completed in 0s 866ms
π
I think I tried both YARN_CACHE_FOLDER
and YARN_VERSION
in my earlier experiments. Maybe I did not set both correct values at the same tome or there is some difference between Netlify UI and the toml file here.
Thanks again β and I hope that Netlify will fix this permanently soon!
Just tried to set up a new Netlify site with a Netlify Build Plugin defined in netlify.toml
and hit this. Disabling Yarn PnP by using nodeLinkers: node-modules
worked but is non-ideal.
@ineffyble do you have a reproducible example? (with the plugin inside the netlify.tom
) So that I can troubleshoot whats happening here?
@lukasholzer Have a look at https://github.com/ineffyble/effy.space/tree/broken - I've just reverted the change in this branch so that you can see the netlify build
failure.
@ineffyble I see yarn Plug'n Play is failing for you inside the plugins installation for local plugins:
https://app.netlify.com/sites/bespoke-meerkat-b0b937/deploys/6377551993b59a000932e6ba
@lukasholzer I'm not quite sure what you mean. Yarn is happy, but Netlify believes the plugin isn't installed.
This happens locally as well for me when running netlify build
with the CLI
Hello @lukasholzer! I just happen to have found this thread and was wondering where are we in the support of Yarn PnP ( and also Corepack)?
Also will Netlify support Yarn v4 that landed few month ago?
The current documentation says that PnP is not supported (https://docs.netlify.com/configure-builds/manage-dependencies/#yarn)
@netlify/build
runsyarn
on two instances, which are both opt-in behavior:@netlify/plugin-functions-install-core
plugin is used, we runyarn
in the functions directory@netlify/plugin-local-install-core
plugin is used, we runyarn
in any local plugins directoryIn both cases, we detect
yarn
by looking for the presence of ayarn.lock
. This does not work in many cases, which is why we might want to also check for the presence of aUSE_YARN
environment, as suggested in https://github.com/netlify/build-image/pull/440Also, the command we are running is
yarn install --no-progress --non-interactive --cache-folder=/opt/buildhome/.yarn_cache
. According to https://github.com/netlify/build-image/pull/440, this has few issues:--cache-folder
was replaced with the environment variableYARN_GLOBAL_FOLDER
in Yarn v2--no-progress
and--non-interactive
do not exist anymore