Open manuq opened 7 years ago
I have the same error with typescript @types definitions that are not added to the offline mirror. If I remove the node_modules, turn off wifi and install --offline everything work but after clearing the cache I get the same error "Can't make a request in offline mode". I'm using Yarn 0.26.
@manuq Yarn 0.26 will log the name of package it failed to fetch now.
I have run into this too. There is an issue with pruning true and scoped dependencies where they are not saved in the offline cache. Take a look at #3509.
@brianhorst Thanks, removing yarn-offline-mirror-pruning true
did the trick.
I think the issue may be with packages beginning with '@' not matching the lock file. The same thing happens with the packages installed by react create app: the package called "@timer/detect-port@1.1.3"
in the lock file corresponds to the offline mirror file @timer-detect-port-1.1.3.tgz (note the hyphen in place of the slash). At the very end of the online install process, the tgz file is removed from the offline mirror folder.
@types
. So @gunta987, worth checking if the issue is general to all packages starting with @
.[2/4] Fetching packages...
verbose 0.8 Performing "GET" request to "https://registry.yarnpkg.com/@types/node/-/node-7.0.29.tgz".
@brianhorst, @kavi87 I can confirm that removing yarn-offline-mirror-pruning true
from .yarnc does the trick, and makes @types-node-7.0.29.tgz file appear in my "Offline mirror" folder. This is for Yarn v0.24.6
Yarn v0.26.1 doesn't have the issue described. So I'm moving my workflow to this version.
Just to confirm, are we in agreement that this is fixed in 0.26.1? If so, can we close this issue since it's already fixed?
This is still happening for me.
sraman@sraman: bar$ yarn add jsdom@11.0.0 --offline
yarn add v0.27.5
info No lockfile found.
[1/4] Resolving packages...
error Couldn't find any versions for "@types/node" that matches "^6.0.46" in our cache. Possible versions: ""
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.
It is definitely available in my cache.
sraman@sraman: bar$ yarn cache ls | grep "@types"
@types/node 6.0.78 npm https://registry.yarnpkg.com/@types/node/-/node-6.0.78.tgz#5d4a3f579c1524e01ee21bf474e6fba09198f470
@types/node 6.0.87 npm https://registry.yarnpkg.com/@types/node/-/node-6.0.87.tgz#5ab5774f8351a33a935099fa6be850aa0b0ad564
@types/node 6.0.88 npm https://registry.yarnpkg.com/@types/node/-/node-6.0.88.tgz#f618f11a944f6a18d92b5c472028728a3e3d4b66
So great to see this resolved!
I remember trying to find an answer to this earlier this year with not resolve, and ended up switching back to npm. Since there was a constant back and forth from all three projects of who's module was causing the issue, there wasn't an answer (especially not this one). I no longer feel dirty being able to use Yarn again.
I did find that I continued to have this issue with 1.3.2, even with the node_modules removed. Once I cleared the cache, the install worked without a hitch and both the yarn-offline-mirror
and yarn-offline-mirror-pruning true
setting set.
I still have this issue with
yarn --version
1.3.2
npm --version
5.5.1
The problem is caused by the cypress
package. I've tried clearing cache, modifying my yarn-offline-mirror-pruning
setting; to no effect. I've a custom registry set though; not sure if it affects anything here.
[2/4] Fetching packages...
verbose 2.45 Error: Can't make a request in offline mode ("https://art-bobcat.autodesk.com:443/artifactory/api/npm/autodesk-npm-virtual/@cypress/listr-verbose-renderer/-/listr-verbose-renderer-0.4.1.tgz")
at RequestManager.request (/usr/local/lib/node_modules/yarn/lib/cli.js:61929:59)
at NpmRegistry.request (/usr/local/lib/node_modules/yarn/lib/cli.js:30362:32)
at /usr/local/lib/node_modules/yarn/lib/cli.js:59335:33
at Generator.next (<anonymous>)
at step (/usr/local/lib/node_modules/yarn/lib/cli.js:92:30)
at /usr/local/lib/node_modules/yarn/lib/cli.js:110:14
at new Promise (<anonymous>)
at new F (/usr/local/lib/node_modules/yarn/lib/cli.js:29389:28)
at /usr/local/lib/node_modules/yarn/lib/cli.js:89:12
at TarballFetcher.fetchFromExternal (/usr/local/lib/node_modules/yarn/lib/cli.js:59395:7)
error Can't make a request in offline mode ("https://art-bobcat.autodesk.com:443/artifactory/api/npm/autodesk-npm-virtual/@cypress/listr-verbose-renderer/-/listr-verbose-renderer-0.4.1.tgz")
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
I think this seems to be happening for all (some?) scoped package tarballs.
yarn offline mirrors appear to be completely broken, at least according to the instructions here https://yarnpkg.com/blog/2016/11/24/offline-mirror/ which appear to be the official documentation
there are 2 different bugs AFAICT:
bug 1. resolving scoped types when the yarn cache is populated. to repro, follow the steps in the offline-mirror blog post, DO NOT clean the cache, add (e.g.) rollup as a dependency, which depends on @types/acorn
, run yarn install --offline
and see this error:
$ yarn install --offline
yarn install v1.5.1
info No lockfile found.
[1/4] Resolving packages...
error An unexpected error occurred: "Couldn't find package \"@types/acorn@^4.0.3\" required by \"rollup@^0.57.1\" on the \"npm\" registry.".
info If you think this is a bug, please open a bug report with the information provided in "/tmp/npm-test/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
Error: Couldn't find package "@types/estree@0.0.38" required by "is-reference@^1.1.0" on the "npm" registry.
at new MessageError (/home/erick/.npm_packages/lib/node_modules/yarn/lib/cli.js:186:110)
at PackageRequest.<anonymous> (/home/erick/.npm_packages/lib/node_modules/yarn/lib/cli.js:36274:17)
at Generator.throw (<anonymous>)
at step (/home/erick/.npm_packages/lib/node_modules/yarn/lib/cli.js:98:30)
at /home/erick/.npm_packages/lib/node_modules/yarn/lib/cli.js:111:13
at <anonymous>
bug 2. locating any offline packages after teh cache has been cleaned: again follow the steps in the offline-mirror blog post, add lodash
as a simple dependency with no transitive dependencies, this time do a yarn cache clean
before attempting the offline install. result:
$ yarn install --offline
yarn install v1.5.1
info No lockfile found.
[1/4] Resolving packages...
error An unexpected error occurred: "Couldn't find any versions for \"lodash\" that matches \"^4.17.5\" in our cache (possible versions are \"\"). This is usually caused by a missing entry in the lockfile, running Yarn without the --offline flag may help fix this issue.".
info If you think this is a bug, please open a bug report with the information provided in "/tmp/npm-test/yarn-error.log".
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
this is with yarn 1.5.1:
$ yarn config list
yarn config v1.5.1
info yarn config
{ 'version-tag-prefix': 'v',
'version-git-tag': true,
'version-git-sign': false,
'version-git-message': 'v%s',
'init-version': '1.0.0',
'init-license': 'MIT',
'save-prefix': '^',
'ignore-scripts': false,
'ignore-optional': false,
registry: 'https://registry.yarnpkg.com',
'strict-ssl': true,
'user-agent': 'yarn/1.5.1 npm/? node/v8.9.1 linux x64',
lastUpdateCheck: 1521455361620,
'yarn-offline-mirror': '/home/erick/npm-packages-offline-cache',
'yarn-offline-mirror-pruning': false }
info npm config
{ prefix: '/home/erick/.npm_packages' }
Done in 0.04s.
@erickj ~I can't seem to reproduce what you are experiencing.~ (see edit below)
~/Projects/yarn-test 🐒 yarn init -y
yarn init v1.5.1
warning The yes flag has been set. This will automatically answer yes to all questions, which may have security implications.
success Saved package.json
✨ Done in 0.04s.
~/Projects/yarn-test 🐒 cat .yarnrc
yarn-offline-mirror "./npm-packages-offline-cache"
yarn-offline-mirror-pruning true
~/Projects/yarn-test 🐒 yarn add rollup
yarn add v1.5.1
info No lockfile found.
[1/4] 🔍 Resolving packages...
[2/4] 🚚 Fetching packages...
[3/4] 🔗 Linking dependencies...
[4/4] 📃 Building fresh packages...
success Saved lockfile.
success Saved 48 new dependencies.
info Direct dependencies
└─ rollup@0.57.1
info All dependencies
├─ @types/acorn@4.0.3
├─ @types/estree@0.0.38
├─ acorn-dynamic-import@3.0.0
├─ acorn@5.5.3
├─ arr-diff@2.0.0
├─ arr-flatten@1.1.0
├─ array-unique@0.2.1
├─ braces@1.8.5
├─ date-time@2.1.0
├─ estree-walker@0.3.1
├─ expand-brackets@0.1.5
├─ expand-range@1.8.2
├─ extglob@0.3.2
├─ filename-regex@2.0.1
├─ fill-range@2.2.3
├─ for-in@1.0.2
├─ for-own@0.1.5
├─ glob-base@0.3.0
├─ glob-parent@2.0.0
├─ irregular-plurals@1.4.0
├─ is-dotfile@1.0.3
├─ is-equal-shallow@0.1.3
├─ is-extendable@0.1.1
├─ is-number@2.1.0
├─ is-posix-bracket@0.1.1
├─ is-primitive@2.0.0
├─ is-reference@1.1.0
├─ isarray@1.0.0
├─ isobject@2.1.0
├─ locate-character@2.0.5
├─ micromatch@2.3.11
├─ normalize-path@2.1.1
├─ object.omit@2.0.1
├─ parse-glob@3.0.4
├─ parse-ms@1.0.1
├─ plur@2.1.2
├─ preserve@0.2.0
├─ pretty-ms@3.1.0
├─ randomatic@1.1.7
├─ regex-cache@0.4.4
├─ remove-trailing-separator@1.1.0
├─ repeat-string@1.6.1
├─ require-relative@0.8.7
├─ rollup-pluginutils@2.0.1
├─ rollup@0.57.1
├─ signal-exit@3.0.2
├─ sourcemap-codec@1.4.1
└─ time-zone@1.0.0
✨ Done in 3.80s.
~/Projects/yarn-test 🐒 rm -rf node_modules/
~/Projects/yarn-test 🐒 yarn install --offline
yarn install v1.5.1
[1/4] 🔍 Resolving packages...
[2/4] 🚚 Fetching packages...
[3/4] 🔗 Linking dependencies...
[4/4] 📃 Building fresh packages...
✨ Done in 0.33s.
and the lodash example
~/Projects/yarn-test 🐒 yarn add lodash
yarn add v1.5.1
[1/4] 🔍 Resolving packages...
[2/4] 🚚 Fetching packages...
[3/4] 🔗 Linking dependencies...
[4/4] 📃 Building fresh packages...
success Saved lockfile.
success Saved 1 new dependency.
info Direct dependencies
└─ lodash@4.17.5
info All dependencies
└─ lodash@4.17.5
✨ Done in 0.72s.
~/Projects/yarn-test 🐒 yarn cache clean
yarn cache v1.5.1
success Cleared cache.
✨ Done in 5.34s.
~/Projects/yarn-test 🐒 rm -rf node_modules/
~/Projects/yarn-test 🐒 yarn install --offline
yarn install v1.5.1
[1/4] 🔍 Resolving packages...
[2/4] 🚚 Fetching packages...
[3/4] 🔗 Linking dependencies...
[4/4] 📃 Building fresh packages...
✨ Done in 1.11s.
I have a Jenkins CI server building one of my projects almost daily using the offline cache (in a clean docker container that doesn't have any global yarn cache) and have never had an issue with it not resolving the packages in the offline cache.
If you look in the ./npm-packages-offline-cache
dir, are the files there?
Edit:
Actually, you might be on to something here... it seems like sometimes the offline cache dir doesn't get all the package files put into it.
~/Projects/yarn-test 🐒 echo "yarn-offline-mirror \"./npm-packages-offline-cache\"" > .yarnrc
~/Projects/yarn-test 🐒 rm -rf node_modules/
~/Projects/yarn-test 🐒 yarn
yarn install v1.5.1
[1/4] 🔍 Resolving packages...
[2/4] 🚚 Fetching packages...
[3/4] 🔗 Linking dependencies...
[4/4] 📃 Building fresh packages...
✨ Done in 0.52s.
~/Projects/yarn-test 🐒 ls npm-packages-offline-cache/
left-pad-1.2.0.tgz
~/Projects/yarn-test 🐒 cat package.json
{
"name": "yarn-test",
"version": "1.0.0",
"main": "index.js",
"license": "MIT",
"dependencies": {
"left-pad": "^1.2.0",
"lodash": "^4.17.5",
"rollup": "^0.57.1"
}
}
despite my project having 3 dependencies, only left-pad
ends up in the offline cache folder, which will certainly cause problems later on during an --offline
install with no cache. 🤔 Now how the heck did that happen?
~/Projects/yarn-test 🐒 yarn cache clean
yarn cache v1.5.1
success Cleared cache.
✨ Done in 0.20s.
~/Projects/yarn-test 🐒 rm -rf node_modules/
~/Projects/yarn-test 🐒 yarn
yarn install v1.5.1
[1/4] 🔍 Resolving packages...
[2/4] 🚚 Fetching packages...
[3/4] 🔗 Linking dependencies...
[4/4] 📃 Building fresh packages...
✨ Done in 1.44s.
~/Projects/yarn-test 🐒 ls npm-packages-offline-cache/
@types-acorn-4.0.3.tgz for-in-1.0.2.tgz is-reference-1.1.0.tgz pretty-ms-3.1.0.tgz
@types-estree-0.0.38.tgz for-own-0.1.5.tgz isarray-1.0.0.tgz randomatic-1.1.7.tgz
acorn-5.5.3.tgz glob-base-0.3.0.tgz isobject-2.1.0.tgz regex-cache-0.4.4.tgz
acorn-dynamic-import-3.0.0.tgz glob-parent-2.0.0.tgz kind-of-3.2.2.tgz remove-trailing-separator-1.1.0.tgz
arr-diff-2.0.0.tgz irregular-plurals-1.4.0.tgz kind-of-4.0.0.tgz repeat-element-1.1.2.tgz
arr-flatten-1.1.0.tgz is-buffer-1.1.6.tgz left-pad-1.2.0.tgz repeat-string-1.6.1.tgz
array-unique-0.2.1.tgz is-dotfile-1.0.3.tgz locate-character-2.0.5.tgz require-relative-0.8.7.tgz
braces-1.8.5.tgz is-equal-shallow-0.1.3.tgz lodash-4.17.5.tgz rollup-0.57.1.tgz
date-time-2.1.0.tgz is-extendable-0.1.1.tgz micromatch-2.3.11.tgz rollup-pluginutils-2.0.1.tgz
estree-walker-0.3.1.tgz is-extglob-1.0.0.tgz normalize-path-2.1.1.tgz signal-exit-3.0.2.tgz
expand-brackets-0.1.5.tgz is-glob-2.0.1.tgz object.omit-2.0.1.tgz sourcemap-codec-1.4.1.tgz
expand-range-1.8.2.tgz is-number-2.1.0.tgz parse-glob-3.0.4.tgz time-zone-1.0.0.tgz
extglob-0.3.2.tgz is-number-3.0.0.tgz parse-ms-1.0.1.tgz
filename-regex-2.0.1.tgz is-posix-bracket-0.1.1.tgz plur-2.1.2.tgz
fill-range-2.2.3.tgz is-primitive-2.0.0.tgz preserve-0.2.0.tgz
So maybe it's hitting the cache and not copying packages to the offline mirror? Except that left-pad was also in my cache but that one was copied into the offline cache the first time... very weird.
Edit 2
Now I can't reproduce the thing I just saw above. The same commands do get all the packages intot he offline cache dir:
~/Projects/yarn-test 🐒 rm -rf npm-packages-offline-cache/
~/Projects/yarn-test 🐒 rm -rf node_modules/
~/Projects/yarn-test 🐒 echo "yarn-offline-mirror \"./npm-packages-offline-cache\"" > .yarnrc
~/Projects/yarn-test 🐒 yarn
yarn install v1.5.1
[1/4] 🔍 Resolving packages...
[2/4] 🚚 Fetching packages...
[3/4] 🔗 Linking dependencies...
[4/4] 📃 Building fresh packages...
✨ Done in 0.52s.
~/Projects/yarn-test 🐒 ls npm-packages-offline-cache/
@types-acorn-4.0.3.tgz fill-range-2.2.3.tgz is-posix-bracket-0.1.1.tgz plur-2.1.2.tgz
@types-estree-0.0.38.tgz for-in-1.0.2.tgz is-primitive-2.0.0.tgz preserve-0.2.0.tgz
acorn-5.5.3.tgz for-own-0.1.5.tgz is-reference-1.1.0.tgz pretty-ms-3.1.0.tgz
acorn-dynamic-import-3.0.0.tgz glob-base-0.3.0.tgz isarray-1.0.0.tgz randomatic-1.1.7.tgz
arr-diff-2.0.0.tgz glob-parent-2.0.0.tgz isobject-2.1.0.tgz regex-cache-0.4.4.tgz
arr-flatten-1.1.0.tgz irregular-plurals-1.4.0.tgz kind-of-3.2.2.tgz remove-trailing-separator-1.1.0.tgz
array-unique-0.2.1.tgz is-buffer-1.1.6.tgz kind-of-4.0.0.tgz repeat-element-1.1.2.tgz
braces-1.8.5.tgz is-dotfile-1.0.3.tgz locate-character-2.0.5.tgz repeat-string-1.6.1.tgz
date-time-2.1.0.tgz is-equal-shallow-0.1.3.tgz lodash-4.17.5.tgz require-relative-0.8.7.tgz
estree-walker-0.3.1.tgz is-extendable-0.1.1.tgz micromatch-2.3.11.tgz rollup-0.57.1.tgz
expand-brackets-0.1.5.tgz is-extglob-1.0.0.tgz normalize-path-2.1.1.tgz rollup-pluginutils-2.0.1.tgz
expand-range-1.8.2.tgz is-glob-2.0.1.tgz object.omit-2.0.1.tgz signal-exit-3.0.2.tgz
extglob-0.3.2.tgz is-number-2.1.0.tgz parse-glob-3.0.4.tgz sourcemap-codec-1.4.1.tgz
filename-regex-2.0.1.tgz is-number-3.0.0.tgz parse-ms-1.0.1.tgz time-zone-1.0.0.tgz
so... I'm not sure what happened when the files didn't end up in the offline cache dir. There must be some odd edge case that causes some packages to get skipped.
I thought I'd note this here as it seems somewhat still ongoing and my issue seems similar.
I seem to be running into a very similar odd edge case. We tried to add the package apidoc - https://github.com/apidoc/apidoc as a dev dependency and it doesn't seem to have added all of the necessary files to the offline cache, but only on Linux.
Running with NodeJs version 10 ./bin/yarn-1.7.0.js install --offline --ignore-platform
on windows 10/Server 2016 everything works fine. Running the same command on Centos 7 results in errors:
error Can't make a request in offline mode ("https://registry.yarnpkg.com/semver/-/semver-5.3.0.tgz"
After manually adding that package to the offline cache, then results in
error Can't make a request in offline mode ("https://registry.yarnpkg.com/entities/-/entities-1.1.1.tgz
Maybe this is something different, but it's interesting that this is only causing problems on our Linux build. I haven't yet been able to try a newer version of yarn so I'm not sure if that would have any impact.
These also seem like dev dependencies of our dev dependencies, which I wouldn't think yarn should care about. Does yarn download all dev dependencies of every package?
(When using yarn 1.3.2 on another machine it does seem to include all missing packages so perhaps there's a bug introduced later causing this)
I have encountered this issue a couple of times too. yarn cache clean
doesn't work for me, but rm -rf node_modules
then yarn install
fixes it.
For anyone else's benefit. After further investigation, I believe our issues have been caused by the yarn global cache under %APPDATA%. It appears if you have packages in there, it won't add them to the configured offline cache. This seems to mostly happen with dependencies of dependencies. Our Linux server doesn't have a proper yarn global cache so it catches any packages not found in our offline cache.
For me, on Linux (Debian 9) it works as it should, but on Windows I have the same issue, as you. Please check the following console log.
C:\Temp\testing-yarn>systeminfo | findstr /B /C:"OS Name" /C:"OS Version"
OS Name: Microsoft Windows 10 Enterprise
OS Version: 10.0.18362 N/A Build 18362
C:\Temp\testing-yarn>yarn --version
1.21.1
C:\Temp\testing-yarn>node --version
v12.14.0
C:\Temp\testing-yarn>yarn config get yarn-offline-mirror
c:\Temp\off\mirror
C:\Temp\testing-yarn>yarn config get yarn-offline-mirror
undefined
C:\Temp\testing-yarn>yarn config set yarn-offline-mirror c:\Temp\testing-yarn\mirror
yarn config v1.21.1
success Set "yarn-offline-mirror" to "c:\\Temp\\testing-yarn\\mirror".
Done in 0.07s.
C:\Temp\testing-yarn>yarn config get yarn-offline-mirror
c:\Temp\testing-yarn\mirror
C:\Temp\testing-yarn>yarn global list
yarn global v1.21.1
Done in 0.11s.
C:\Temp\testing-yarn>yarn global add lodash
yarn global v1.21.1
[1/4] Resolving packages...
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
warning "lodash@4.17.15" has no binaries
Done in 7.05s.
C:\Temp\testing-yarn>ls mirror
lodash-4.17.15.tgz
C:\Temp\testing-yarn>yarn global remove lodash
yarn global v1.21.1
[1/2] Removing module lodash...
[2/2] Regenerating lockfile and installing missing dependencies...
success Uninstalled packages.
Done in 0.51s.
C:\Temp\testing-yarn>yarn global list
yarn global v1.21.1
Done in 0.09s.
C:\Temp\testing-yarn>yarn cache clean
yarn cache v1.21.1
success Cleared cache.
Done in 0.45s.
C:\Temp\testing-yarn>yarn global add lodash --offline
yarn global v1.21.1
[1/4] Resolving packages...
error Couldn't find any versions for "lodash" that matches "latest" in our cache (possible versions are ""). This is usually caused by a missing entry in the lockfile, running Yarn without the --offline flag may help fix this issue.
info Visit https://yarnpkg.com/en/docs/cli/global for documentation about this command.
C:\Temp\testing-yarn>yarn add lodash --offline
yarn add v1.21.1
info No lockfile found.
[1/4] Resolving packages...
error Couldn't find any versions for "lodash" that matches "latest" in our cache (possible versions are ""). This is usually caused by a missing entry in the lockfile, running Yarn without the --offline flag may help fix this issue.
info Visit https://yarnpkg.com/en/docs/cli/add for documentation about this command.
Do you want to request a feature or report a bug? Bug.
What is the current behavior? Yarn fails to install Electron from the "Offline mirror" folder. Fails with "Can't make a request in offline mode".
If the current behavior is a bug, please provide the steps to reproduce.
The exception is:
This is my .yarnrc:
What is the expected behavior? According to this blog post,
yarn install --offline
should work after cleaning the yarn cache. But with Electron, it tries to fetch some package. Adding--verbose
doen't help, it does not show what package is trying to fetch.Please mention your node.js, yarn and operating system version.