yarn probably shouldn't cache packages resolved with a file path

[donaldpipowitch]

Do you want to request a feature or report a bug?

I guess a bug.

What is the current behavior? If the current behavior is a bug, please provide the steps to reproduce.

Say you have the following project structure:

component-foo/
└── package.json
└── index.js

yarn-test/
└── package.json

With the following files:

component-foo/package.json:

{
  "name": "component-foo",
  "version": "1.0.0",
  "private": true,
  "main": "index.js"
}

component-foo/index.js:

console.log('foo');

yarn-test/package.json:

{
  "name": "yarn-test",
  "version": "1.0.0",
  "private": true,
  "dependencies": {
    "component-foo": "file:../component-foo"
  }
}

Now you run $ yarn install inside yarn-test/ and yarn-test/node_modules/component-foo/index.js is:

console.log('foo');

Now remove yarn-test/node_modules/ and yarn-test/yarn.lock and change component-foo/index.js to this:

console.log('bar');

Now you run $ yarn install inside yarn-test/ again and yarn-test/node_modules/component-foo/index.js will be:

console.log('foo');

It uses the cached version of component-foo, but component-foo/index.js has changed.

What is the expected behavior?

I'd expect that yarn-test/node_modules/component-foo/index.js should be this at the end:

console.log('bar');

Maybe packages installed with local paths like file:../ shouldn't be cached at all, if we can't figure out, if they have changed.

(FYI: It looks like npm doesn't cache them.)

Please mention your node.js, yarn and operating system version.

$ node -v
v6.9.1

$ yarn -V
0.18.0

macOS 10.12.1

[gregjacobs]

@jonathantorley I just tried it again with --force and it still does not work with yarn 1.12.3

[gregjacobs]

For others facing this issue: one workaround that I used to actually get this to work was to increase the version number of the dependent package. A bit annoying, and you have to remember to do it on each change (unless you can script it).

yarn should definitely not require this workaround when installing local packages

[ohtangza]

I added yarn upgrade MY_PACKAGE_NAME in preinstall script, and it upgrades to the latest NPM version well. (Still, I need to bump the NPM version manually).

[leavesster]

It seems yarn add file:PATH now always upgrade the new content now in yarn 1.13.0. yarn install still not.

[joshleblanc]

@leavesster Still doesn't for me.

I have to rename the tgz in order for it to update

[nboliver]

Try using the link command: https://yarnpkg.com/lang/en/docs/cli/link/

[elpupi]

yarn add file:PATH did not upgrade the new content for me. Moreover, files in package.json and .npmignore are not respected.

It works of you change the version of your package.

If you want yarn add file:PATH to respect files in package.json and .npmignore, you must run yarn pack in your local package dependency and then where you want to install it run yarn add file:path-to-local-pacckage.tgz

[elpupi]

Try using the link command: https://yarnpkg.com/lang/en/docs/cli/link/

yarn link is not intended to answer the same problem. It is not good if someone wants the npm package like if it was published respecting the files in package.json and .npmignore

[leavesster]

@leavesster Still doesn't for me.

I have to rename the tgz in order for it to update

I don't have any tgz in my package which I use yarn add file: PACKAGE_PAH to add it in the current project. I just have js code in my the package. Maybe tgz still wrong ?

[SumNeuron]

not working for me either

[wKich]

@bestander why this issue closed? Yarn still caching local files and tgz packages. Workspaces is not a solution in some cases, for example if one package have deps with specific versions, and other package have devDeps same packages as the first, but with another versions, and if you link one package into another, your project have been broken.

I have same issue as @gregjacobs. yarn cache clean package didn't help. But I figured out that if you install yarn add path/to/package.tgz and then replace archive to another one, you can install new version simply change the path like yarn add path/to/../to/package.tgz, so absolute path is same, but for yarn its different.

I don’t understand where else yarn store cached packages by resolution path, even yarn cache list --pattern package is empty.

[wKich]

I think the problem is somewhere here https://github.com/yarnpkg/yarn/blob/eb2b565bb9b948e87b11119482ebc184a9d66141/src/resolvers/exotics/tarball-resolver.js#L58-L63

Whats happening:

• From url path/to/package.tgz generates hash (that's why path/to/package.tgz and path/to/../to/package.tgz results to different hashes)
• With that hash creates temp directory for package (like this one /Users/kich/Library/Caches/Yarn/v4/.tmp/c019816ee7d10ed5e1fef4072e8cc617)
• For first run isValidModuleDest return false and all goes right
• Tarball packages extracted in this directory
• Files from that directory installed into the project
• You modify develop project sources and build/pack the new package tarball with same name and location that previous one
• And you try install new tarball, but generated hash is a same as before and temp directory was not cleared after first run
• So in this time isValidModuleDest return true
• And yarn install old version of your package
• You run yarn cache clean package, but temp directory remains untouched

[wKich]

@bestander can we simply remove temp directory here https://github.com/yarnpkg/yarn/blob/master/src/config.js#L431 ?

[leozzyzheng]

Facing the same problem, the cache under temp folder takes 10GB after I work on my local pacakge for only one day!

[ghost]

Can we reopen this issue? We are facing the same problem. Two projects referencing another package via file:. After some builds in our CI/CD server, the cache folder is starting to occupy a lot of space.

[alxtz]

I think the link protocol is the best fix for this, file:// doesn't work since it still requires manually clean caches & force installing

https://github.com/yarnpkg/yarn/issues/2165#issuecomment-345825904

just make your dependency something like this

"<package>": "link:./libs/<package>"

[wKich]

@alxtz does links protocol works with packages in tgz?

[silasdavis]

I think the link protocol is the best fix for this, file:// doesn't work since it still requires manually clean caches & force installing

#2165 (comment)

just make your dependency something like this

"<package>": "link:./libs/<package>"

Thanks for this this replicates the behaviour of NPM which will symlink file:.. references in node_modules. It does not appear to be documented, at least not here where I would expect to find it: https://yarnpkg.com/lang/en/docs/package-json/

[ojboj]

Sadly, link can't be used in all cases.

For example, I need a shared/peer dependency from my SDK package, which when I make changes, I would link for local dev work. With link, yarn does not realise that the dependency is a shared/peer dependency and uses the local package where the symlink is, which is incorrect.

I've been using yarn pack alongside yarn add file:<path_to_packed_tgz> to get around this. Whilst I can continue to rename the package every time I pack it and paste it into my repo, so as not to generate the same hash, as per @wKich fantastic find, it's damn annoying.

I forked the repo and put an extra clause in the if statement to stop yarn from ever loading a local .tgz package from its cache if it's specified using yarn add file:<path>.

const dest = this.config.getTemp(crypto.hash(url));
// If specified using file: never load from cache
if (!url.match(/file:/) && (await this.config.isValidModuleDest(dest))) {
  // load from local cache
} else {
  // continue as if it's a new package
}

I can make a PR if people want it, though I've never done that before and it's a pretty hacky fix. Could someone confirm if this approach would continue to add the local packages to yarn's cache or not, please?

[souporserious]

@ojboj for now, yalc might help until this gets a proper fix. It has been working really well for me to test packages locally before publishing.

[ojboj]

@souporserious That is exactly what I needed/wanted, thank you so much!

[bobber205]

Insane this is still an issue after so many years.

[wKich]

Is it fixed in yarn@2.x?

[souporserious]

@wKich I believe so! I haven't personally tested it, but it looks like they solve local development through the new portal protocol.

[garyo]

I still get "unpack in same destination" error using the link: protocol, and it's referencing my cache dir, so it seems to me yarn is still caching link: dependencies. I'm referencing 2 copies of the same local package, copied into the .deps/ folder of each app that uses it -- front-end and renderer in the error below. (Can't link to the original for unrelated reasons)

warning Pattern ["@horizon/common@link:packages/front-end/.deps/@horizon/common"] is trying to unpack in the same destination "/home/garyo/.cache/yarn/v6/npm-@horizon-common-0.1.0/node_modules/@horizon/common" as pattern ["@horizon/common@link:packages/renderer/.deps/@horizon/common"]. This could result in non-deterministic behavior, skipping.

[thejohnfreeman]

This issue is not fixed by the link: and portal: protocols for those of us like @ojboj and me who install packages from a tarball instead of a directory, e.g. file:path/to/dependency.tgz. One annoying workaround is to use a throwaway cache directory like yarn install --cache-folder $(mktemp -d). My preferred workaround is to just bust the cache for every offline dependency: rm -rf $(yarn cache dir)/.tmp. Thanks to @wKich for the detective work.

[geoidesic]

Why is this closed? I still get this issue in Yarn 1.22.19... Yarn caches local packages, which is not helpful for development.

[justsaul]

Believe it or not, it's still an issue with yarn berry (3+). I wish there were better alternatives to deal with devDeps.

[hoegge]

It is actually 4+ years https://github.com/yarnpkg/yarn/issues/6037

[sickdyd]

When developing a package with React 18 I had various issues using yarn link, so I just rely on building a local package and installing in my React app, this issue is pretty annoying... there should be some way to forcefully reinstalling the single local package ignoring the cache instead of using yarn cache clean.