renovatebot / renovate

Home of the Renovate CLI: Cross-platform Dependency Automation by Mend.io
https://mend.io/renovate
GNU Affero General Public License v3.0
17.34k stars 2.27k forks source link

Error updating Yarn binary #16733

Closed rarkins closed 2 years ago

rarkins commented 2 years ago

How are you running Renovate?

Mend Renovate hosted app on github.com

If you're self-hosting Renovate, tell us what version of Renovate you run.

No response

Please select which platform you are using if self-hosting.

No response

If you're self-hosting Renovate, tell us what version of the platform you run.

No response

Was this something which used to work for you, and then stopped?

I never saw this working

Describe the bug

I am seeing quite a lot of these errors in the hosted app.

Example repo: https://github.com/Darkflame72/SMS-Discord-Bot in this PR: https://github.com/Darkflame72/SMS-Discord-Bot/pull/7

Seems to have increased recently: image

Relevant debug logs

Logs ``` {"level":20,"branch":"renovate/yarn-3.x","msg":"getBranchPr(renovate/yarn-3.x)","time":"2022-07-24T09:12:55.903Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"findPr(renovate/yarn-3.x, undefined, open)","time":"2022-07-24T09:12:55.903Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Found PR #7","time":"2022-07-24T09:12:55.904Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"branchExists=true","time":"2022-07-24T09:12:55.904Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"dependencyDashboardCheck=undefined","time":"2022-07-24T09:12:55.904Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"PR rebase requested=true","time":"2022-07-24T09:12:55.904Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Checking if PR has been edited","time":"2022-07-24T09:12:55.904Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Found existing branch PR","time":"2022-07-24T09:12:55.905Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Checking schedule(at any time, null)","time":"2022-07-24T09:12:55.905Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"No schedule defined","time":"2022-07-24T09:12:55.905Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Setting current branch to main","time":"2022-07-24T09:12:55.905Z"} {"level":20,"branch":"renovate/yarn-3.x","branchName":"main","latestCommitDate":"2022-07-24T21:07:00+12:00","msg":"latest commit","time":"2022-07-24T09:12:56.257Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Manual rebase requested via Dependency Dashboard","time":"2022-07-24T09:12:56.370Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Using reuseExistingBranch: false","time":"2022-07-24T09:12:56.371Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"manager.getUpdatedPackageFiles() reuseExistinbranch=false","time":"2022-07-24T09:12:56.376Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"npm.updateDependency(): packageManager.yarn = 3.2.2","time":"2022-07-24T09:12:56.628Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Updating yarn in package.json","time":"2022-07-24T09:12:56.637Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Updated 1 package files","time":"2022-07-24T09:12:56.638Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Getting updated lock files","time":"2022-07-24T09:12:56.639Z"} {"level":20,"branch":"renovate/yarn-3.x","packageFiles":["package.json"],"msg":"Writing package.json files","time":"2022-07-24T09:12:56.640Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Writing any updated package files","time":"2022-07-24T09:12:56.641Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Writing package.json","time":"2022-07-24T09:12:56.641Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"npmrc file found in repository","time":"2022-07-24T09:12:56.670Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Writing updated .npmrc file to .npmrc","time":"2022-07-24T09:12:56.670Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Generating yarn.lock for .","time":"2022-07-24T09:12:56.671Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Spawning yarn install to create yarn.lock","time":"2022-07-24T09:12:56.672Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Enabling global cache as zero-install is not detected","time":"2022-07-24T09:12:56.673Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"No node constraint found - using latest","time":"2022-07-24T09:12:56.674Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Updating Yarn binary","time":"2022-07-24T09:12:56.674Z"} {"level":20,"branch":"renovate/yarn-3.x","image":"node","msg":"Using docker to execute","time":"2022-07-24T09:12:56.675Z"} {"level":20,"branch":"renovate/yarn-3.x","toolName":"corepack","resolvedVersion":"0.12.1","msg":"Resolved stable matching version","time":"2022-07-24T09:12:56.690Z"} {"level":20,"branch":"renovate/yarn-3.x","image":"docker.io/renovate/node","msg":"No tag or tagConstraint specified","time":"2022-07-24T09:12:56.692Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Fetching Docker image: docker.io/renovate/node","time":"2022-07-24T09:12:56.692Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"Finished fetching Docker image docker.io/renovate/node@sha256:dfc6f5d8a32593133be29f098d53f36a49ef270b58eec97dcf10cc230ad37894","time":"2022-07-24T09:13:18.693Z"} {"level":20,"branch":"renovate/yarn-3.x","command":"docker run --rm --name=renovate_node --label=renovate_child -v \"/mnt/renovate/gh/Darkflame72/SMS-Discord-Bot\":\"/mnt/renovate/gh/Darkflame72/SMS-Discord-Bot\" -v \"/tmp/renovate-cache\":\"/tmp/renovate-cache\" -e NPM_CONFIG_CACHE -e npm_config_store -e CI -e YARN_ENABLE_IMMUTABLE_INSTALLS -e YARN_HTTP_TIMEOUT -e YARN_GLOBAL_FOLDER -e YARN_ENABLE_GLOBAL_CACHE -e BUILDPACK_CACHE_DIR -w \"/mnt/renovate/gh/Darkflame72/SMS-Discord-Bot\" docker.io/renovate/node bash -l -c \"install-tool corepack 0.12.1 && yarn set version 3.2.2 && yarn install --mode=update-lockfile\"","msg":"Executing command","time":"2022-07-24T09:13:18.871Z"} {"level":20,"branch":"renovate/yarn-3.x","cmd":"docker run --rm --name=renovate_node --label=renovate_child -v \"/mnt/renovate/gh/Darkflame72/SMS-Discord-Bot\":\"/mnt/renovate/gh/Darkflame72/SMS-Discord-Bot\" -v \"/tmp/renovate-cache\":\"/tmp/renovate-cache\" -e NPM_CONFIG_CACHE -e npm_config_store -e CI -e YARN_ENABLE_IMMUTABLE_INSTALLS -e YARN_HTTP_TIMEOUT -e YARN_GLOBAL_FOLDER -e YARN_ENABLE_GLOBAL_CACHE -e BUILDPACK_CACHE_DIR -w \"/mnt/renovate/gh/Darkflame72/SMS-Discord-Bot\" docker.io/renovate/node bash -l -c \"install-tool corepack 0.12.1 && yarn set version 3.2.2 && yarn install --mode=update-lockfile\"","durationMs":175369,"stdout":"installing v2 tool corepack v0.12.1\nnpm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead.\n\nadded 1 package in 1s\nlinking tool corepack v0.12.1\n0.12.1\nInstalled v2 /usr/local/buildpack/tools/v2/corepack.sh in 2 seconds\n➤ YN0000: Retrieving https://repo.yarnpkg.com/3.2.2/packages/yarnpkg-cli/bin/yarn.js\n➤ YN0000: Saving the new release in .yarn/releases/yarn-3.2.2.cjs\n➤ YN0000: Done in 1s 476ms\n➤ YN0000: ┌ Resolution step\n➤ YN0000: └ Completed in 0s 714ms\n➤ YN0000: ┌ Fetch step\n➤ YN0013: │ 32 packages were already cached, 4 had to be fetched\n➤ YN0000: └ Completed in 2m 42s\n➤ YN0000: ┌ Link step\n➤ YN0073: │ Skipped due to mode=update-lockfile\n➤ YN0000: └ Completed\n➤ YN0000: Done with warnings in 2m 43s\n","stderr":"","msg":"exec completed","time":"2022-07-24T09:16:14.062Z"} {"level":20,"branch":"renovate/yarn-3.x","msg":"yarn.lock needs updating","time":"2022-07-24T09:16:14.162Z"} {"level":20,"branch":"renovate/yarn-3.x","resolvedPaths":[".yarn/cache",".pnp.cjs",".pnp.js",".pnp.loader.mjs"],"msg":"updateYarnOffline resolvedPaths","time":"2022-07-24T09:16:14.337Z"} {"level":50,"branch":"renovate/yarn-3.x","err":{"code":"ERR_INVALID_ARG_TYPE","message":"The \"path\" argument must be of type string. Received undefined","stack":"TypeError [ERR_INVALID_ARG_TYPE]: The \"path\" argument must be of type string. Received undefined\n at new NodeError (internal/errors.js:322:7)\n at validateString (internal/validators.js:124:11)\n at Object.join (path.js:1148:7)\n at Object.join (/home/ubuntu/renovateapp/node_modules/upath/build/code/upath.js:51:33)\n at updateYarnBinary (/home/ubuntu/renovateapp/node_modules/renovate/dist/modules/manager/npm/post-update/index.js:349:49)\n at async getAdditionalFiles (/home/ubuntu/renovateapp/node_modules/renovate/dist/modules/manager/npm/post-update/index.js:530:44)\n at async processBranch (/home/ubuntu/renovateapp/node_modules/renovate/dist/workers/repository/update/branch/index.js:304:33)\n at async writeUpdates (/home/ubuntu/renovateapp/node_modules/renovate/dist/workers/repository/process/write.js:25:21)\n at async update (/home/ubuntu/renovateapp/node_modules/renovate/dist/workers/repository/process/extract-update.js:109:15)\n at async Object.renovateRepository (/home/ubuntu/renovateapp/node_modules/renovate/dist/workers/repository/index.js:43:25)\n at async renovateRepository (/home/ubuntu/renovateapp/app/worker/index.js:310:26)\n at async /home/ubuntu/renovateapp/app/worker/index.js:570:5"},"msg":"Error updating Yarn binary","time":"2022-07-24T09:16:14.508Z"} {"level":20,"branch":"renovate/yarn-3.x","updatedArtifacts":["yarn.lock"],"msg":"Updated 1 lock files","time":"2022-07-24T09:16:14.511Z"} ```

Have you created a minimal reproduction repository?

No reproduction, but I have linked to a public repo where it occurs

Gabriel-Ladzaretti commented 2 years ago

Tried to reproduce it with this repro with no success. The spike in errors seems related to the recent yarn 3.2.2 release (3 days ago), as we are using it to update the yarn.lock. from the changelogs, this might be related.

rarkins commented 2 years ago

I think the updates may be missing the vendored extra files?

bodinsamuel commented 2 years ago

Hey feel free to dismiss if this is not related but since today every NPM update are blocked with this error This script must be run as root (example pr)

Screenshot 2022-07-25 at 18 25 28
rarkins commented 2 years ago

@bodinsamuel that's probably unrelated but very concerning. Can you check the logs to see if there's any other useful info there?

bodinsamuel commented 2 years ago

JobID 750412317

```sh DEBUG: Executing command(branch="renovate/pin-dependencies") { "command": "docker run --rm --name=renovate_node --label=renovate_child -v \"/mnt/renovate/gh/bodinsamuel/altheia\":\"/mnt/renovate/gh/bodinsamuel/altheia\" -v \"/tmp/renovate-cache\":\"/tmp/renovate-cache\" -e NPM_CONFIG_CACHE -e npm_config_store -e CI -e YARN_ENABLE_IMMUTABLE_INSTALLS -e YARN_HTTP_TIMEOUT -e YARN_GLOBAL_FOLDER -e YARN_ENABLE_GLOBAL_CACHE -e BUILDPACK_CACHE_DIR -w \"/mnt/renovate/gh/bodinsamuel/altheia\" docker.io/renovate/node:10.13.0 bash -l -c \"install-tool corepack 0.12.1 && yarn install --mode=update-lockfile\"" } DEBUG: rawExec err(branch="renovate/pin-dependencies") { "err": { "name": "ExecError", "cmd": "/bin/sh -c docker run --rm --name=renovate_node --label=renovate_child -v \"/mnt/renovate/gh/bodinsamuel/altheia\":\"/mnt/renovate/gh/bodinsamuel/altheia\" -v \"/tmp/renovate-cache\":\"/tmp/renovate-cache\" -e NPM_CONFIG_CACHE -e npm_config_store -e CI -e YARN_ENABLE_IMMUTABLE_INSTALLS -e YARN_HTTP_TIMEOUT -e YARN_GLOBAL_FOLDER -e YARN_ENABLE_GLOBAL_CACHE -e BUILDPACK_CACHE_DIR -w \"/mnt/renovate/gh/bodinsamuel/altheia\" docker.io/renovate/node:10.13.0 bash -l -c \"install-tool corepack 0.12.1 && yarn install --mode=update-lockfile\"", "stderr": "", "stdout": "This script must be run as root\n", "options": { "cwd": "/mnt/renovate/gh/bodinsamuel/altheia", "encoding": "utf-8", "env": { "NPM_CONFIG_CACHE": "/tmp/renovate-cache/others/npm", "npm_config_store": "/tmp/renovate-cache/others/pnpm", "CI": "true", "YARN_ENABLE_IMMUTABLE_INSTALLS": "false", "YARN_HTTP_TIMEOUT": "100000", "YARN_GLOBAL_FOLDER": "/tmp/renovate-cache/others/berry", "YARN_ENABLE_GLOBAL_CACHE": "1", "HOME": "/home/ubuntu", "PATH": "/home/ubuntu/.npm-global/bin:/usr/local/node/14.18.1/bin:/home/ubuntu/.local/bin:/usr/local/python/3.9.3/bin:/home/ubuntu/bin:/home/ubuntu/renovateapp/node_modules/.bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "LC_ALL": "C.UTF-8", "LANG": "C.UTF-8", "BUILDPACK_CACHE_DIR": "/tmp/renovate-cache/buildpack" }, "maxBuffer": 10485760, "timeout": 900000 }, "exitCode": 1, "message": "Process exited with exit code \"1\"", "stack": "ExecError: Process exited with exit code \"1\"\n at ChildProcess. (/home/ubuntu/renovateapp/node_modules/renovate/dist/util/exec/common.js:85:24)\n at ChildProcess.emit (events.js:412:35)\n at ChildProcess.emit (domain.js:475:12)\n at Process.ChildProcess._handle.onexit (internal/child_process.js:282:12)" } } ```
viceice commented 2 years ago

it's using node v10.13.0? looks like wrong constraints?

rarkins commented 2 years ago

Yes in the .nvmrc

viceice commented 2 years ago

🤔 is yarn v3.2 compatible with node v10? that repo is mixing a lot different configuration options. engines and package manager differ for requested yarn version

bodinsamuel commented 2 years ago

aah will try, sorry for the noise then 🙏🏻

ylemkimon commented 2 years ago

With Corepack, Yarn 2+ can work without checking in the binary file. As a result, yarnPath in .yarnrc.yml may be undefined, or even .yarnrc.yml may not exist. And this will be the default in Yarn 4: https://github.com/yarnpkg/berry/pull/4254. https://github.com/renovatebot/renovate/blob/dd5ca94a03cdfb5d3db21e66ba0d4f7de2500786/lib/modules/manager/npm/post-update/index.ts#L426-L428

I think there were some logics depending on the existence of .yarnrc.yml, so they should be updated as well.

renovate-release commented 2 years ago

:tada: This issue has been resolved in version 32.146.0 :tada:

The release is available on:

Your semantic-release bot :package::rocket: