Extracting the tar content of undefined failed

opiation commented 6 years ago

Do you want to request a feature or report a bug? Reporting a bug when running yarn install to install node dependencies. For severity, this bug seems critical considering it essentially prevents me from acquiring node dependencies.

What is the current behavior? Fails sometimes with errors like the following:

yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
error https://registry.yarnpkg.com/lodash/-/lodash-4.17.10.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod '/usr/local/share/.cache/yarn/v2/npm-lodash-4.17.10-1b7793cf7259ea38fb3661d4d38b3260af8ae4e7/_cacheHas.js'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
error https://registry.yarnpkg.com/lodash/-/lodash-4.17.10.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "EEXIST: file already exists, mkdir '/usr/local/share/.cache/yarn/v2/npm-lodash-4.17.10-1b7793cf7259ea38fb3661d4d38b3260af8ae4e7'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
error https://registry.yarnpkg.com/fbjs/-/fbjs-0.8.17.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod '/usr/local/share/.cache/yarn/v2/npm-fbjs-0.8.17-c4d598ead6949112653d6588b01a5cdcd9f90fdd/lib/resolveImmediate.js'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command

The occurrence of this error is the challenging part. It does not always fail and does not always fail with the same dependency. The installation is sometimes successful after 3-5 tries.

If the current behavior is a bug, please provide the steps to reproduce. I've attempted to install dependencies on bare-metal and in a node:8-alpine docker container. Both can sometimes encounter the error. I've tested this on my personal device in Montreal, Canada (Mac OS X10.13), on a AWS EC2 instance (Ubuntu 18.04), on a GCE instance (Ubuntu 16.04) and on a production server in France (Debian 8). Each of them can sometimes encounter this error. I've also tried installing with and without yarn.lock to no avail. Find a package.json that appears to sometimes reproduce the issue in this gist. The issue does not seem to happen with projects having fewer dependencies.

What is the expected behavior? Successful installation of all packages like npm install or npm ci which deterministically succeeds without any tar or caching errors.

Please mention your node.js, yarn and operating system version. Tested with the following version: Node: 8 LTS, 10 Yarn: 1.9.2, 1.9.4 OS: Ubuntu 18.04 LTS, Ubuntu 16.04 LTS, Debian 8, Mac OSX 10.13 Registrie: registry.yarnpkg.com, registry.npmjs.org, private registry

If you require any additional information, don't hesitate to request it. FWIW, reducing the network-concurrency seems to produce a slightly higher success ratio but not consistently enough to conclude the errors are related. It may be an area to investigate however. I've unfortunately exhausted all the time I could afford to spend on this after a few days of troubleshooting. I've reluctantly had to migrate all our CI builds back to using npm install/npm ci :(

Titozzz commented 6 years ago

Same issue, it's blocking my CI too, we updated recently to yarn 1.9.2

adrienharnay commented 6 years ago

@opiation The error is indeed random but I may have found the cause: do you have distant git URLs in your package.json without .git at the end? We had two of those and adding .git fixed the issue. Not sure why the error message doesn't directly state this is is the problem though.

adrienharnay commented 6 years ago

Also, maybe related: https://github.com/yarnpkg/yarn/issues/5536

opiation commented 6 years ago

@adrienharnay, can you define what you mean by distant? For the record, here's the package.json I used. There's only one github dependency and I still get errors without it. I'm not sure how I would append .git to non-git dependencies, unless I've misunderstood your suggestion.

adrienharnay commented 6 years ago

Distant wasn't the right word, I just meant packages installed from Git 🙂

Could you try this?

"storybook-addon-markdown": "https://github.com/mihalik/storybook-addon-markdown.git"

opiation commented 6 years ago

As per my previous comment, I still encounter the issue without storybook-addon-markdown dependency. Thus, I don't believe this issue stems from yarn improperly handling git URLs.

adrienharnay commented 6 years ago

Indeed, I read too quickly. Well, that fixed our bug, but I have no idea about yours 😕 Sorry

Titozzz commented 6 years ago

@opiation Did you update you yarn.lock file too? Cause I had to do that

opiation commented 6 years ago

@Titozzz, I encounter this error with and without the yarn.lock file. I have deleted and recreated the lock file several times to no avail.

ekarlso commented 6 years ago

I get this too and I dont have any packages from git.

luwes commented 6 years ago

I wanted to get around this issue (https://github.com/yarnpkg/yarn/issues/6256) by using the tarball versions of the packages but indeed the error above is thrown for tarball urls on self-hosted Github enterprise.

github.com hosted tarballs somehow did work. e.g. https://github.com/luwes/chameleon/archive/grasshopper-v0.0.1-beta.4.tar.gz

kvnify commented 6 years ago

I'm seeing the same issue with a project we have. However when I remove the deps that run a prepare script as part of the install (due to being git urls) then it works. These happen to be pointing git urls, but I think it's actually the prepare that seem to kick off more yarn install processes that seem to subvert the mutex flag for some reason. I wonder if that's because the other processes are kicked off by the root process rather than by different root processes. I don't know if this information helps or if its actually way off base. But I figured I would share what I've found regardless.

adrienharnay commented 6 years ago

@khendry I got the issue again, and you are right, it comes from git dependencies which have a prepare script in their package.json! :+1:

opnet commented 6 years ago

I've been tracking this down with a project we have and so far narrowed it down to the concurrent install the git-fetcher starts here. If the package that is being installed by the git-fetcher has any of the same dependencies of the currently installing package a race condition is created where the duplicated packages will be untarred to offline cache at the same time.

I haven't seen enough of the code base to understand where/what the correct fix is, but that's the start of the issue.

bjentsch commented 6 years ago

Any news on this? We're facing this issue too.

idexter commented 6 years ago

Same problem. It's impossible to use yarn with CI. Every 2nd build failed with this error 😞

holyxiaoxin commented 6 years ago

try delete node_modules,

yarn cache clean
yarn install --network-concurrency 1

muuki88 commented 6 years ago

Thanks for sharing this. It's at least a workaround 🤗 , but no real solution if you want your build time reasonable fast 😅

kvnify commented 6 years ago

We did try using the --network-concurrency flag with no success. So that does not actually resolve this particular issue. The flag addresses concurrency at a higher level than where the issue occurs.

idexter commented 6 years ago

For me --network-concurrency 1 solves the problem. I know it temporary fix, but it works. But value must be exactly 1.

kvnify commented 6 years ago

I spoke too soon. I had asked my team mate if we'd tried this, rather than actually trying myself and he was very confident that we had...he was mistaken and miss-read the previous post thinking it was related to the mutex flag, not network concurrency. We've since re-tried and can confirm this does seem to also address the issue for us.

hulkish commented 6 years ago

setting --network-concurrency 1 does not actually work for me.

right now, the only workaround I've come across involves completely regenning yarn.lock. The error I get is:

2.054 Performing "GET" request to "https://<private-artifactory-npm-registry>/@myorg/eslint-plugin-import/-/@myorg/eslint-plugin-import-3.0.0.tgz".
verbose 2.519 Error: https://<private-artifactory-npm-registry>/@myorg/eslint-plugin-import/-/@myorg/eslint-plugin-import-3.0.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
    at MessageError.ExtendableBuiltin (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:237:66)
    at new MessageError (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:266:123)
    at Extract.<anonymous> (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:59446:14)
    at emitOne (events.js:121:20)
    at Extract.emit (events.js:211:7)
    at Extract.module.exports.Extract.destroy (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:135306:17)
    at Extract.module.exports.Extract._final (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:135364:34)
    at callFinal (/Users/me/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:70270:10)
    at _combinedTickCallback (internal/process/next_tick.js:139:11)
    at process._tickCallback (internal/process/next_tick.js:181:9)

Update: I've just found that using --skip-integrity-check allows me to bypass this error. While obviously that's really a solution. This looks like kind of an important bug in the integrity check logic.

Im using node@8.12.0, yarn@1.12.0

hulkish commented 6 years ago

@arcanis @rally25rs more details on this error:

So, this seems pretty odd to me that it's failing integrity checksum, considering the sha1's are the same:

Error: sha1-Sl7Hxk364iw6FBJNus3uhG2Ay8Q= integrity checksum failed when using sha1: wanted sha1-Sl7Hxk364iw6FBJNus3uhG2Ay8Q= but got sha1-AHoWKXweP+Pg9aZkGBsAjFruGaM=. (77 bytes)
    at Transform.on (/Users/shargrove/.nvm/versions/node/v8.12.0/lib/node_modules/yarn/lib/cli.js:32831:19)
    at emitNone (events.js:111:20)
    at Transform.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1064:12)
    at _combinedTickCallback (internal/process/next_tick.js:139:11)
    at process._tickCallback (internal/process/next_tick.js:181:9)

Update: After seeing above, I confirmed that --skip-integrity-check bypasses this error. Looks like a more serious bug in integrity check logic.

hulkish commented 6 years ago

@opiation out of curiosity, can you paste your package.json? Are you using the following "override" technique anywhere?

"dependencies": {
  "foo": "npm:@myorg/foo"
}

For ex, I'm using it:

"eslint-plugin-import": "npm:@myorg/eslint-plugin-import",

And, this is the package I see the error for.. So, I wonder if this is related?

opiation commented 6 years ago

@hulkish, as per my initial post, here's the gist I created with my package.json, yarn.lock and all the tests I ran that all produced the error described. To clarify, each line in failing_test.sh can encounter this error but not consistently. They may have to be tried more than once to encounter the error. Just to have it in this thread, I'll summarize each test below:

Failing tests

yarn install
yarn install --frozen-lockfile
yarn install --pure-lockfile
yarn install --mutex network
yarn install --network-concurrency 1
All of the above tests with rm yarn.lock beforehand
All of the above tests in a node:alpine container with git installed (alpine at the time this thread was created)
All of the above tests in a node:8-alpine container with git installed

As for the "override" technique, I am uncertain what you mean. If you are referring to the protocol-like prefix in the dependency value (like npm: in your example), then yes, one dev dependency uses a github package:

"storybook-addon-markdown": "github:mihalik/storybook-addon-markdown"

The errors are still encountered however even when I remove the dev dependency so this does not seem related.

theandrewlane commented 6 years ago

Shout out to @holyxiaoxin - adding --network-concurrency 1 resolved this for my CI 👍

arcanis commented 6 years ago

ping @imsnif ? It seems related to the integrity check, per @hulkish's comment

Aminelahlou commented 6 years ago

@khendry Not using prepare anymore on our git dependencies resolved this for our ci, while --network-concurrency 1, --child-concurrency 1 and --skip-integrity-check were not enough

schneidmaster commented 6 years ago

We were able to fix this with npm config set always-auth true (as documented here). As best I can tell, npm by default will provide your credentials only for publishing packages, not for fetching them. For some reason yarn previously didn’t respect that setting, but it now does.

mvonballmo commented 6 years ago

I recently had this issue, using yarn 1.12.3 and node 10.13.0. After trying many of the solutions above, to no avail, deleting/regenerating the yarn.lock file worked.

OmgImAlexis commented 6 years ago

I've been getting a similar issue. Removing the yarn.lock like @mvonballmo suggested was the only thing that got it running. It's still not fully working though.

yarn install v1.12.3
[1/4] Resolving packages...
[2/4] Fetching packages...
error https://registry.yarnpkg.com/iconv-lite/-/iconv-lite-0.4.23.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOSPC: no space left on device, write"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

yarn install v1.12.3
info No lockfile found.
[1/4] Resolving packages...
warning celebrate > joi > hoek@5.0.4: This version is no longer maintained. Please upgrade to the latest version.
warning xo > eslint > file-entry-cache > flat-cache > circular-json@0.3.3: CircularJSON is in maintenance only, flatted is its successor.
[2/4] Fetching packages...
error https://registry.yarnpkg.com/babel-runtime/-/babel-runtime-6.26.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOSPC: no space left on device, write"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

imsnif commented 6 years ago

Hi friends,

So, judging by the different errors reported here - this actually seems like it can potentially be several different issues: ENOSPC: no space left on device, write, wanted sha1-Sl7Hxk364iw6FBJNus3uhG2Ay8Q= but got sha1-AHoWKXweP+Pg9aZkGBsAjFruGaM= (btw, takes a hard look, but these are not the same), the file appears to be corrupt: "Unexpected end of data", etc.

While I appreciate these can be happening in a similar place, they might be caused by entirely different issues and/or environments. The integrity check (specifically the untarStream on error callback - thanks for the detailed debug @hulkish!) is a funnel that can gather many errors, and it's a little hard to give feedback beyond the actual error to the user.

The above is especially true for the integrity migration (populating an old-style yarn.lock with the new integrity fields), since this one-time process (one time assuming it succeeds ofc) is more network intensive than a normal install (it loops through all packages without an integrity field and fetches their registry manifest).

The theories about a race condition are interesting and very definitely a possibility, I'd be happy to look into them further. I'm afraid though that @opiation 's reproduction did not work for me. I'm now running my 7th local installation of it and it still works without issue (I did not run the script, rather I just ran yarn to install it with that package.json and yarn.lock - I understand this still caused the issue for you?)

@opiation - can you still reproduce this issue? Under the same conditions? Maybe we can go down a level of resolution and you can tell me everything you do down to the commands to make this happen?

Anyone else on this thread has a setup they can share that reproduces this issue even partially consistently? I'd be very happy to get to the bottom of this.

JeremyTCD commented 6 years ago

I encountered the same error message in my CI system:

2018-11-12T04:32:13.0386630Z error https://pkgs.dev.azure.com/JeremyTCD/_packaging/Main/npm/registry/cheerio/-/cheerio-0.22.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
2018-11-12T04:32:20.4838361Z 
2018-11-12T04:32:20.4852626Z     yarn install v1.12.3                                                                    
2018-11-12T04:32:20.4853491Z     [1/4] Resolving packages...                                                             
2018-11-12T04:32:20.4855400Z     [2/4] Fetching packages...                                                              
2018-11-12T04:32:20.4856037Z     info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Managed to figure it out my particular issue though. Thought I'd leave a note here for anyone who encounters something similar:

Cause

I called yarn install on my local machine after adding a new dependency to my project (cheerio@0.22.0). Due to a local .npmrc, yarn restored the dependency from a private registry of mine. The generated yarn.lock contained the following lines:

cheerio@0.22.0:
  version "0.22.0"
  resolved "https://pkgs.dev.azure.com/JeremyTCD/_packaging/Main/npm/registry/cheerio/-/cheerio-0.22.0.tgz#a9baa860a3f9b595a6b81b1a86873121ed3a269e"
  dependencies:
  ...

Note how the package was resolved from a private repository. On my CI machine, I didn't have a .npmrc with the credentials for the private registry. This was the cause of the error message:

https://pkgs.dev.azure.com/JeremyTCD/_packaging/Main/npm/registry/cheerio/-/cheerio-0.22.0.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"

I fixed my local .npmrc and regenerated my yarn.lock:

cheerio@0.22.0:
  version "0.22.0"
  resolved "http://registry.npmjs.org/cheerio/-/cheerio-0.22.0.tgz#a9baa860a3f9b595a6b81b1a86873121ed3a269e"
  integrity sha1-qbqoYKP5tZWmuBsahocxIe06Jp4=

Note how the package is now resolved from the default NPM registry. The error stopped ocurring once I did this.

Fix

If the cause of your issue is the same as mine, you can:

Add required credentials to your CI machine or
Tweak your local .npmrc (yarn config list will print the registry that yarn restores from), then regenerate yarn.lock.

Notes

Perhaps the error message could be more specific.

Edit: Initially thought rolling back Yarn would solve the issue - accidentally linked my faulty commit to this issue. Yarn wasn't the issue in the end.

gregorybolkenstijn commented 6 years ago

TL;DR: try deleting your yarn.lock file and generating it again.

I got an error when trying to build on Netlify: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"

Deleting the node_modules folder and the yarn.lock file and then generating them again via yarn install got me a new yarn.lock file with different dependencies. With this new file Netlify succesfully built my project.

oderby commented 6 years ago

@imsnif agreed that there are seemingly multiple different issues being reported here. I believe I have a repro case from a project I'm working on that triggers the problem outlined by @khendry here

I'm seeing the same issue with a project we have. However when I remove the deps that run a prepare script as part of the install (due to being git urls) then it works. These happen to be pointing git urls, but I think it's actually the prepare that seem to kick off more yarn install processes that seem to subvert the mutex flag for some reason. I wonder if that's because the other processes are kicked off by the root process rather than by different root processes.

Sharing repro steps below in the hopes it allows you to reproduce the problem. Let me know if you need more information.

Repro steps

With node v10.3.0 and yarn v1.12.3, in a new test folder, download the package.json and yarn.lock from this gist
run rm -rf ~/.cache/yarn* node_modules/ && yarn install --frozen-lockfile --network-concurrency 16 (clear cache and previously install node modules for reliable environment. Set concurrency high to increase chances of hitting issue)

observe output like the following:

yarn install v1.12.3
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
warning Pattern ["object-assign@latest"] is trying to unpack in the same destination "/home/ocderby/.cache/yarn/v4/npm-object-assign-4.1.1-2109adc7965887cfc05cbbd442cac8bfbb360863/node_modules/object-assign" as pattern ["object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4","object-assign@^4.1.1","object-assign@^4.1.0","object-assign@4.x","object-assign@^4.1.0","object-assign@^4.1.1","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.0","object-assign@^4.1.1","object-assign@^4.1.1","object-assign@^4.0.1","object-assign@^4.0.1","object-assign@^4.1.0","object-assign@^4.0.1","object-assign@^4.0.1","object-assign@^4.0.1","object-assign@^4.1.0","object-assign@^4.0.1"]. This could result in non-deterministic behavior, skipping.
info No lockfile found.
[1/4] Resolving packages...
warning eslint > file-entry-cache > flat-cache > circular-json@0.3.3: CircularJSON is in maintenance only, flatted is its successor.
warning jest > jest-cli > prompts > kleur@2.0.2: Please upgrade to kleur@3 or migrate to 'ansi-colors' if you prefer the old syntax. Visit <https://github.com/lukeed/kleur/releases/tag/v3.0.0\> for migration path(s).
[2/4] Fetching packages...
error https://registry.yarnpkg.com/lodash/-/lodash-4.17.4.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod '/home/ocderby/.cache/yarn/v4/npm-lodash-4.17.4-78203a4d1c328ae1d86dca6460e369b57f4055ae/node_modules/lodash/_shortOut.js'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

Other notes

I've tried a variety of things, here are my notes:

The issue does not reproduce 100% of the time for me. As noted above, increasing the network concurrency used seems to make the issue more likely to occur.
Using a version of react-textarea-autosize published on the package registry makes the problem go away (seeming to confirm what @khendry reported above)
Setting --mutex file doesn't seem to help here at all
As reported above, if I limit network concurrency to 1 (via --network-concurrency 1 argument), everything installs properly, albeit slower.
I've reproduced this on node v8.12.0, with both yarn v1.9.4 and v1.12.3. This was running on the circleci/node:8-stretch docker image running on Circle CI 2.0.

ankurk91 commented 5 years ago

I start seeing this error recently after updating yarn to 1.12.3. See my travis-ci build failure https://travis-ci.org/ankurk91/vue-cleave-component

$ yarn install --non-interactive
yarn install v1.12.3
[1/5] Validating package.json...
[2/5] Resolving packages...
[3/5] Fetching packages...
error https://registry.yarnpkg.com/har-validator/-/har-validator-5.1.2.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
The command "yarn install --non-interactive" failed and exited with 1 during .

It is happening with only with har-validator@5.1.2. I will post back if i got success somehow. PS. It was specific to the har-validator package.

garaboncias commented 5 years ago

I also get error https://registry.yarnpkg.com/har-validator/-/har-validator-5.1.2.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Unexpected end of data" with curl i got 404 for https://registry.yarnpkg.com/har-validator/-/har-validator-5.1.2.tgz but in my browser i can download it. One of my server if i downgrade yarn to 1.12.1 it start to work but on the other server even if i downgrade the error remains the same (I remove yarn cache dir in both case). Is that possible that is some kind of cloudflare (config) issue?

arcanis commented 5 years ago

No, this specific instance (yours and the one by @ankurk91) are caused by har-validator having been unpublished (cf #6694).

jhalborg commented 5 years ago

I get this error in my CI environment only, after having added another repo as a dependency ("@team/myproject": "git+ssh://git@bitbucket.org/team/myproject.git#master",) . I can confirm that

adding --network-concurrency 1 to my CI script solves the issue, but of course makes the build very slow
running yarn install --network-concurrency 16 provokes the error locally as well

Neither cleaning cache nor resetting yarn.lock made a difference for me

EDIT: It seems the --network-concurrency 1 fix is not consistent, unfortunately 😢

webmobiles commented 5 years ago

same error here, easy to reproduce: yarn upgrade typescript@^2.8

then: yarn upgrade typescript@2.8

I did ctrl+c while installing this last package.. and when I try 'yarn upgrade' again I get:


yarn upgrade v1.12.3
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
error https://registry.yarnpkg.com/typescript/-/typescript-2.8.4.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, stat '/Users/u/Library/Caches/Yarn/v4/npm-typescript-2.8.4-0b1db68e6bdfb0b767fa2ab642136a35b059b199/node_modules/typescript/lib/lib.d.ts'"
info Visit https://yarnpkg.com/en/docs/cli/upgrade for documentation about this command.

csvan commented 5 years ago

UPDATE: The below was due to corrupt metadata in our Sonatype Nexus install, and is hence not a Yarn issue. Leaving for context.

Seeing this for multiple packages in our CI environment. Yarn 1.12.3 and Node 11.1:

responsive-props-1.2.2.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?"

styled-components-breakpoint-2.1.3.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "Invalid tar header. Maybe the tar is corrupted or it needs to be gunzipped?"

jbergens commented 5 years ago

I had a similar problem but I get ....the file appears to be corrupt: "EBUSY: ...". I cleared my whole yarn cache an re-ran it and still got the same error so it seems like yarn is creating files and locking them for itself.

This is on Windows 10.

yarn install v1.10.1 [1/4] Resolving packages... [2/4] Fetching packages... error https://registry.yarnpkg.com/fbjs/-/fbjs-0.8.17.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "EBUSY: resource busy or locked, open 'c:\\src\\yarn\\cache\\v2\\npm-fbjs-0.8.17-c4d598ead6949112653d6588b01a5cdcd9f90fdd\\lib\\UserAgent.js'" info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

I did a workaround by running "yarn --pnp" which worked. Strange since that should be newer and probably more unstable code.

davidalee commented 5 years ago

Removing yarn.lock was what got it working for me.

binchik commented 5 years ago

Hey everyone, just had the same issue. Resolved by removing .npmrc from home dir.

rm ~/.npmrc

nothingismagick commented 5 years ago

@binchik - this is the only thing that worked for me.

dhogborg commented 5 years ago

Thank you @binchik, that did the trick for me. 👍

nothingismagick commented 5 years ago

So after rewinding back to the series of events leading up to yarn failing, I believe I executed an npm script in a package.json that was something like this:

"audit": "npm audit"

Which is totally silly, because I never use npm in that project. After this command everything (including npm) would just start to have random faults and never complete, in line with the experience of others in this thread.

arcanis commented 5 years ago

If someone reproducing the error could investigate and figure out what exactly is causing the problem it would be super helpful! I've tried but I can't reproduce it 🙁

Some pointers:

We need to figure what goes into untarStream when it fails - my hypothesis is that maybe we're trying to process a json response as a tarball (https://github.com/yarnpkg/yarn/blob/master/src/fetchers/tarball-fetcher.js#L146-L150)
the only thing I'm think that could matter in the .npmrc is the auth token. I would appreciate if someone could confirm that the problem disappear simply by removing the auth token line from the .npmrc (rather than the whole file)

deleteme commented 5 years ago

FWIW, I encountered this problem today. A few things:

Removing .npmrc fixed it. The only thing in the file had to do with the auth token.
npm install failed too, and it logged a 401 unauthorized error.
After removing the .npmrc file, npm install worked again.

hulkish commented 5 years ago

@deleteme per my findings, this sounds more like a byproduct of the bug, rather than the cause.

I've encountered with and without a .npmrc or .yarnrc

yarnpkg / yarn