search

yarnpkg / yarn

The 1.x line is frozen - features and bugfixes now happen on https://github.com/yarnpkg/berry
https://classic.yarnpkg.com
Other
41.39k stars 2.72k forks source link

should have a resolved reference #4187

Open zeqk opened 7 years ago

zeqk commented 7 years ago

Do you want to request a feature or report a bug? BUG

What is the current behavior? Execute yarn install, produce this error:

should have a resolved reference

If the current behavior is a bug, please provide the steps to reproduce.

yarn-error.log.txt

What is the expected behavior?

No error

Please mention your node.js, yarn and operating system version. Yarn 0.27.5 Node 6.2.1 Windows 8.1 x64

yarn.error.log

Arguments: C:\Program Files (x86)\nodejs\node.exe C:\Program Files (x86)\Yarn\bin\yarn.js import

PATH: C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE\;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn\;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files\TortoiseGit\bin;C:\Program Files\Microsoft Windows Performance Toolkit\;C:\Program Files (x86)\Windows Resource Kits\Tools\;C:\Program Files (x86)\Microsoft Emulator Manager\1.0\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files (x86)\nodejs\;C:\Program Files (x86)\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\ManagementStudio\;C:\Program Files\dotnet\;C:\Program Files\Git\cmd;C:\Program Files (x86)\Yarn\bin;C:\ProgramData\Oracle\Java\javapath;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\VSShell\Common7\IDE\;C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn\;C:\Program Files\Microsoft\Web Platform Installer\;C:\Program Files\TortoiseGit\bin;C:\Program Files\Microsoft Windows Performance Toolkit\;C:\Program Files (x86)\Windows Resource Kits\Tools\;C:\Program Files (x86)\Microsoft Emulator Manager\1.0\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Windows Kits\10\Windows Performance Toolkit\;C:\Program Files (x86)\nodejs\;C:\Program Files (x86)\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\ManagementStudio\;C:\Program Files\dotnet\;C:\Program Files\Git\cmd;C:\Program Files (x86)\Android\android-sdk\build-tools\23.0.2;C:\Users\Usuario\AppData\Local\Yarn\bin

Yarn version: 0.27.5

Node version: 6.2.1

Platform: win32 x64

npm manifest: { "name": "bats", "version": "2.0.0", "description": "Bootstrap 4 + Angular 2 + Typescript", "author": "octubre softlab", "license": "MIT", "dependencies": { "@.angular/common": "2.4.7", "@.angular/compiler": "2.4.7", "@.angular/core": "2.4.7", "@.angular/forms": "2.4.7", "@.angular/http": "2.4.7", "@.angular/platform-browser": "2.4.7", "@.angular/platform-browser-dynamic": "2.4.7", "@.angular/platform-server": "2.4.7", "@.angular/router": "3.4.7", "@.angularclass/conventions-loader": "1.0.13", "@.angularclass/hmr": "1.2.2", "@.angularclass/hmr-loader": "3.0.2", "amcharts3": "github:amcharts/amcharts3", "ammap3": "github:amcharts/ammap3", "animate.css": "3.5.2", "bootstrap": "4.0.0-alpha.4", "chart.js": "2.3.0", "chartist": "0.10.1", "ckeditor": "4.6.0", "core-js": "2.4.1", "d3": "3.5.5", "d3-cloud": "1.2.4", "easy-pie-chart": "2.1.7", "font-awesome": "4.7.0", "fullcalendar": "2.9.1", "google-maps": "3.2.1", "http-proxy": "1.14.0", "http-server": "0.9.0", "ie-shim": "0.1.0", "ionicons": "2.0.1", "is-electron-renderer": "2.0.1", "jqcloud2": "2.0.2", "jquery": "2.2.4", "jquery-slimscroll": "1.3.8", "justgage": "1.2.2", "leaflet": "0.7.7", "leaflet-map": "0.2.1", "lodash": "4.17.4", "moment": "2.17.1", "ng2-bootstrap": "1.3.3", "ng2-charts": "1.4.1", "ng2-ckeditor": "1.1.5", "ng2-completer": "1.1.0", "ng2-smart-table": "0.5.3-0", "ng2-tree": "2.0.0-alpha.3", "ngx-uploader": "2.2.0", "normalize.css": "4.2.0", "reflect-metadata": "0.1.9", "rxjs": "5.0.2", "tether": "1.4.0", "zone.js": "0.7.7" }, "devDependencies": { "@.angular/compiler-cli": "2.4.7", "@.types/d3": "3.5.38", "@.types/electron": "1.4.32", "@.types/fullcalendar": "2.7.37", "@.types/hammerjs": "2.0.34", "@.types/jasmine": "2.2.34", "@.types/jquery": "2.0.40", "@.types/jquery.slimscroll": "1.3.30", "@.types/lodash": "ts2.0", "@.types/node": "6.0.63", "@.types/source-map": "0.5.0", "@.types/uglify-js": "2.6.28", "@.types/webpack": "2.2.5", "add-asset-html-webpack-plugin": "1.0.2", "angular2-template-loader": "0.6.0", "assets-webpack-plugin": "3.5.1", "awesome-typescript-loader": "3.0.4", "bootstrap-loader": "2.0.0-beta.21", "codelyzer": "2.0.0", "copy-webpack-plugin": "4.0.1", "css-loader": "0.26.1", "electron": "1.6.0", "es6-promise": "4.0.5", "es6-shim": "0.35.3", "es7-reflect-metadata": "1.6.0", "exports-loader": "0.6.3", "expose-loader": "0.7.3", "extract-text-webpack-plugin": "2.0.0-rc.3", "file-loader": "0.10.0", "find-root": "1.0.0", "font-awesome-sass-loader": "1.0.3", "gh-pages": "0.12.0", "html-webpack-plugin": "2.28.0", "imports-loader": "0.7.0", "json-loader": "0.5.4", "ng-router-loader": "2.1.0", "ngc-webpack": "1.2.0", "node-sass": "4.5.0", "npm-run-all": "4.0.1", "optimize-js-plugin": "0.0.4", "postcss-loader": "1.3.1", "raw-loader": "0.5.1", "resolve-url-loader": "1.6.1", "rimraf": "2.5.4", "sass-loader": "4.1.1", "script-ext-html-webpack-plugin": "1.7.1", "source-map-loader": "0.1.6", "string-replace-loader": "1.0.5", "style-loader": "0.13.1", "to-string-loader": "1.1.5", "ts-helpers": "1.1.2", "ts-node": "2.1.0", "tslint": "4.4.2", "typedoc": "0.5.6", "typescript": "2.0.10", "url-loader": "0.5.7", "webpack": "2.2.0", "webpack-dev-middleware": "1.9.0", "webpack-dev-server": "2.2.0", "webpack-dll-bundles-plugin": "1.0.0-beta.5", "webpack-merge": "2.6.1", "jasmine-core": "2.4.1", "karma": "1.2.0", "karma-jasmine": "1.0.2", "karma-phantomjs-launcher": "1.0.2", "karma-sourcemap-loader": "0.3.7", "karma-webpack": "1.8.0", "null-loader": "0.1.1", "phantomjs-prebuilt": "2.1.7", "protractor": "4.0.14" }, "scripts": { "rimraf": "rimraf", "tslint": "tslint", "typedoc": "typedoc", "webpack": "webpack --progress --profile --bail", "webpack-dev-server": "webpack-dev-server", "webdriver-manager": "webdriver-manager", "clean": "npm cache clean && npm run rimraf -- node_modules doc coverage dist compiled dll", "clean:dist": "npm run rimraf -- dist", "clean:dll": "npm run rimraf -- dll", "clean:electron": "npm run rimraf -- build", "preclean:install": "npm run clean", "clean:install": "npm set progress=false && npm install", "preclean:start": "npm run clean", "clean:start": "npm start", "watch": "npm run watch:dev", "watch:dev": "npm run build:dev -- --watch", "watch:dev:hmr": "npm run watch:dev -- --hot", "watch:prod": "npm run build:prod -- --watch", "build": "npm run build:dev", "prebuild:dev": "npm run clean:dist", "build:dev": " npm run clean:dist && npm run webpack -- --config config/webpack.dev.js", "prebuild:prod": "npm run clean:dist", "build:prod": "npm run clean:dist && webpack --config config/webpack.prod.js --progress --profile", "server": "npm run server:dev", "server:dev": "webpack-dev-server --config config/webpack.dev.js --progress --profile --watch --content-base src/", "server:dev:hmr": "npm run server:dev -- --inline --hot", "server:prod": "http-server dist -c-1 --cors -P http://localhost:988", "server:test": "http-server dist-demo -c-1 --cors", "server:prod:ci": "http-server dist -p 3000 -c-1 --cors -P http://localhost:988", "webdriver:update": "npm run webdriver-manager update", "webdriver:start": "npm run webdriver-manager start", "lint": "npm run tslint \"src/*/.ts\" --force", "ngc": "./node_modules/.bin/ngc-w -p tsconfig.webpack.json", "pree2e": "npm run webdriver:update -- --standalone", "pretest": "npm run lint", "docs": "npm run typedoc -- --options typedoc.json --exclude '*/.spec.ts' ./src/", "gh-pages": "wintersmith build -C docs && gh-pages -d docs/build", "start": "npm run server:dev", "start:hmr": "npm run server:dev:hmr", "version": "npm run build", "postversion": "git push && git push --tags", "build:electron": "npm run build:electron.full", "prebuild:electron.full": "npm run clean:electron", "build:electron.full": "npm run build:electron.renderer && npm run build:electron.main", "postbuild:electron.full": "npm run electron:start", "build:electron.renderer": "npm run webpack -- --config config/electron/webpack.renderer.prod.js", "build:electron.main": "npm run webpack -- --config config/electron/webpack.electron.prod.js", "electron:start": "electron build", "build:aot:prod": "npm run clean:dist && npm run clean:aot && webpack --config config/webpack.prod.js --progress --profile --bail", "build:aot": "npm run build:aot:prod", "clean:aot": "npm run rimraf -- compiled", "build:ci": "npm run build:prod && npm run build:aot", "test": "karma start", "test:tdd": "npm test -- --no-single-run --auto-watch", "protractor": "protractor", "protractor:delay": "sleep 3 && npm run protractor", "protractor:live": "protractor --elementExplorer", "e2e:live": "npm-run-all -p -r server:prod:ci protractor:live", "e2e": "npm-run-all -p -r server:prod:ci protractor", "ci:aot": "npm run lint && npm run build:aot && npm run e2e", "ci:jit": "npm run lint && npm run build:prod && npm run e2e" }, "engines": { "node": ">= 5.4.1 < 7" } }

yarn manifest: No manifest

Lockfile: No lockfile

Trace: Invariant Violation: should have a resolved reference at invariant (C:\Program Files (x86)\Yarn\lib\yarn-cli.js:1132:15) at ImportPackageRequest.resolveToExistingVersion (C:\Program Files (x86)\Yarn\lib\yarn-cli.js:22853:5) at ImportPackageResolver.resolvePackagesWithExistingVersions (C:\Program Files (x86)\Yarn\lib\yarn-cli.js:57347:11) at C:\Program Files (x86)\Yarn\lib\yarn-cli.js:76480:16 at next (native) at step (C:\Program Files (x86)\Yarn\lib\yarn-cli.js:91:30) at C:\Program Files (x86)\Yarn\lib\yarn-cli.js:102:13 at run (C:\Program Files (x86)\Yarn\lib\yarn-cli.js:87239:22) at C:\Program Files (x86)\Yarn\lib\yarn-cli.js:87252:28 at flush (C:\Program Files (x86)\Yarn\lib\yarn-cli.js:86710:9)

batjko commented 6 years ago

I'm having the same issue with yarn import, but there seems to be virtually nothing on the interwebz about this.

Any news on this?

arcanis commented 6 years ago

We unfortunately can't do much without reproduction script 😐

batjko commented 6 years ago

@zeqk I have created a package.json from your original post above (except I had to remove all the private modules) and all works fine.

So at least we know that the issue is likely related to the private modules.

@arcanis Could it be that yarn import has an issue accessing private repos?

BYK commented 6 years ago

@batjko that's a bit unlikely. I'd try to reproduce this with the latest Yarn and post a minimal reproduction case if possible.

rasor commented 6 years ago

I have same problem with yarn import. Just doing yarn succeeded with yarn v1.1.0

moberhuber commented 6 years ago

Attached yarn-4187.zip is a reproducer.

Using nvs to run node-6.11.4 on Ubuntu 17.04, the following fails:

unzip yarn-4187.zip
cd yarn-4187
nvs auto
npm install   #gets stuff from npm-shrinkwrap.json
node yarn-1.3.2.js import

This is the output that I see:

yarn import v1.3.2
warning package.json: No license field
success Folder in sync.
warning No license field
error An unexpected error occurred: "should have a resolved reference".

The reproducer is from a project of mine, which was migrated from node-0.12 with lots of old libraries, keeping the old versions around with npm-shrinkwrap.json. I'd like to migrate to yarn while keeping the same locked-down versions ... doing just "yarn install" gets loads of updates, so having the "yarn import" workflow actually work would unblock me.

katanacrimson commented 6 years ago

Also encountering this on a yarn import.

npm ls output (good god github, this should be a vertical scrollbar!):

npm_ls.output.txt

And from our dependencies/devDependencies: 

 "dependencies": {
    "@types/angular-ui-router": "^1.1.36",
    "angular": "^1.6.1",
    "angular-clipboard": "^1.6.1",
    "angular-drag-and-drop-lists": "^1.4.0",
    "angular-messages": "^1.6.1",
    "angular-resource": "^1.6.1",
    "angular-sanitize": "^1.6.1",
    "angular-ui-router": "^1.0.0-rc.1",
    "core-js": "^2.4.1",
    "jquery": "^3.1.1",
    "jsonformatter": "^0.6.0",
    "moment": "^2.15.1",
    "papaparse": "^4.3.6",
    "rxjs": "^5.4.0",
    "weakmap": "0.0.6"
  },
  "devDependencies": {
    "@types/angular": "^1.6.2",
    "@types/angular-mocks": "^1.5.11",
    "@types/es6-shim": "^0.31.32",
    "@types/jasmine": "^2.5.40",
    "@types/jquery": "^2.0.39",
    "angular-mocks": "^1.6.1",
    "asap": "^2.0.5",
    "body-parser": "^1.16.0",
    "canonical-path": "0.0.2",
    "dgeni": "^0.4.2",
    "dgeni-packages": "^0.16.4",
    "es6-shim": "^0.35.3",
    "express": "^4.14.0",
    "get-own-enumerable-property-symbols": "^1.0.1",
    "glob": "^7.1.1",
    "gulp": "^3.9.1",
    "gulp-autoprefixer": "^3.1.1",
    "gulp-clean": "^0.3.2",
    "gulp-color": "0.0.1",
    "gulp-concat": "^2.6.0",
    "gulp-cssnano": "^2.1.2",
    "gulp-debug": "^3.0.0",
    "gulp-htmlmin": "^3.0.0",
    "gulp-if": "^2.0.2",
    "gulp-jshint": "^2.0.1",
    "gulp-live-server": "0.0.30",
    "gulp-newer": "^1.3.0",
    "gulp-ng-annotate": "^2.0.0",
    "gulp-ng-html2js": "^0.2.2",
    "gulp-open": "^2.0.0",
    "gulp-protractor": "^3.0.0",
    "gulp-rename": "^1.2.2",
    "gulp-rev": "^7.1.2",
    "gulp-rev-replace": "^0.4.3",
    "gulp-sass": "^2.3.2",
    "gulp-shell": "^0.5.2",
    "gulp-sourcemaps": "^1.7.3",
    "gulp-typescript": "^3.1.6",
    "gulp-uglify": "^2.0.0",
    "gulp-useref": "^3.1.2",
    "intl": "^1.2.5",
    "is-regexp": "^1.0.0",
    "jasmine-core": "^2.5.2",
    "jasmine-jquery": "^2.1.1",
    "jasmine-spec-reporter": "^3.2.0",
    "jshint": "^2.9.3",
    "jshint-stylish": "^2.2.1",
    "karma": "^1.3.0",
    "karma-chrome-launcher": "^2.0.0",
    "karma-coverage": "^1.1.1",
    "karma-jasmine": "^1.0.2",
    "karma-ng-html2js-preprocessor": "^1.0.0",
    "karma-phantomjs-launcher": "^1.0.2",
    "karma-typescript": "^3.0.4",
    "lodash": "^4.17.4",
    "multer": "^1.2.0",
    "ncp": "^2.0.0",
    "ng-metadata": "^4.0.1",
    "ng-table": "^3.0.1",
    "node-mkdirp": "0.0.1",
    "phantomjs": "^2.1.7",
    "portastic": "^1.0.1",
    "protractor": "^5.0.0",
    "recursive-readdir-sync": "^1.0.6",
    "reflect-metadata": "^0.1.9",
    "replacestream": "^4.0.2",
    "request": "^2.75.0",
    "request-promise": "^4.1.1",
    "run-sequence": "^1.2.2",
    "shortid": "^2.2.6",
    "systemjs": "^0.20.2",
    "systemjs-builder": "^0.16.1",
    "systemjs-plugin-text": "0.0.9",
    "tslint": "^4.5.1",
    "tsv": "^0.2.0",
    "typescript": "^2.4.0",
    "weakmap": "0.0.6"
  },

This is a showstopper from us converting to yarn, unfortunately.

yarn 1.3.2, node 6.9.2, Windows 10.

ED: Converted npm ls output to a text file and attached. That's absurdly long. :|

katanacrimson commented 6 years ago

This also fails on yarn 1.4 RC with "Invariant Violation: should have a resolved reference", pointing to yarn/lib/cli.js:1614

ED: This guy's where it's throwing:

https://github.com/yarnpkg/yarn/blob/18ea344707694a02987f111e3d32085893437d9f/src/package-request.js#L200-L213

katanacrimson commented 6 years ago

Ran breakpointing, stepping through resolveToExistingVersion and I see normalizePattern() being called with the correct pattern canonical-path@0.0.2, returning the correct object of { name: "canonical-path", range: "~0.0.2", hasVersion: true }. The transpiled code then stores this to a var called _normalizePattern2, and then rolls on to the next lines, setting consts range and name.

Stepping over each line, when we step onto the line setting name = _normalizePattern2.name, it is being modified, and _normalizePattern2.name now contains "extend" instead of "canonical-path".

Turns out that was because of breakpointing issues of mine. However, I am finding that extends is where yarn import begins failing for me, where inside of this:

https://github.com/yarnpkg/yarn/blob/18ea344707694a02987f111e3d32085893437d9f/src/package-resolver.js#L416-L445

...the call to semver.maxSatisfying() is giving us null. The versionNumbers being passed in are ['3.0.1', '3.0.1'], and the range is '3.0.0'...but I haven't taken the time to start taking apart the semver call or getStrictResolvedPattern.

Not sure where to go from here, but this is a huge blocker for me right now. Is there anything that I can provide in terms of info to help get to the bottom of this, @BYK, @arcanis?

katanacrimson commented 6 years ago

After a lot of troubleshooting, I've basically figured out that you'll need to yarn add the exact dependencies of each dependency in your tree (every single one) into a side project to prime the yarn cache with every package you'll need, and to use yarn import --offline. It seems that registry fetches that occur seem to taint the state of what's loaded in from the node_modules that we're trying to import from, so offline helps immensely.

I also had to work around #5340, which if you have anything that uses fsevents and are on windows, will block your import. Hand-massaging package.json for those deps allowed the import to continue. A horrible, hacky solution, but it worked nonetheless.