Parcel 2 doesn't detect changes in linked dependencies.

[samuelgozi]

🐛 bug report

When working on a dependency locally by linking it, Parcel 2 doesn't detect changes and therefore doesn't rebuild. You also need to remove the cache manually in order for it to work on re-runs.

🎛 Configuration (.babelrc, package.json, cli command)

Package.json

{
    "name": "parcel-tamplate",
    "version": "1.1.0",
    "description": "My personal Parcel Template",
    "main": "public/index.html",
    "source": "src/index.html",
    "author": "Samuel Elgozi <samu.elgozi@gmail.com>",
    "license": "none",
    "private": true,
    "scripts": {
        "dev": "parcel src/index.html",
        "build": "parcel build src/index.html"
    },
    "targets": {
        "main": {
            "includeNodeModules": true
        }
    },
    "devDependencies": {
        "@babel/plugin-proposal-class-properties": "^7.8.3",
        "babel-eslint": "^10.0.2",
        "eslint": "^6.0.1",
        "less": "^3.9.0",
        "parcel": "^2.0.0-alpha.3.2"
    },
    "dependencies": {
        "firebase-auth-lite": "^0.4.1"
    }
}

The commands I use:

parcel src/index.html

🤔 Expected Behavior

Like in version 1, Parcel should detect changes in linked dependencies, invalidate the cache and rebuild.

😯 Current Behavior

No changes are made, which require manual removal of .parcel-cache and then building again.

🔦 Context

I use Parcel to work on npm modules often. Without detecting the changes in linked packages there is no reason for me to use parcel. In fact, it becomes painful to do so...

🌍 Your Environment

Software	Version(s)
Parcel	2.0.0-alpha.3.2
Node	v13.10.1
Yarn	1.22.1
Operating System	MacOs Catalina 10.15.3

[DeMoorJasper]

Could you create a repo with some instructions on how I'd reproduce this? Would help a lot in fixing this

[samuelgozi]

Ok, here you go: https://github.com/samuelgozi/Parcel2-issue-4332 Further instructions can be found in the repo's README file.

[DeMoorJasper]

@samuelgozi thanks for creating the repo, I can reproduce it pretty consistently, will try and figure out why it happens and hopefully fix it

[samuelgozi]

@DeMoorJasper Thank you!

[kalaschnik]

I develop a local npm package and experience the same issue using the npm link functionality. Is there any news on this?

[devongovett]

This is because the file watcher only watches for changes in the project root. It would be somewhat hard to implement support for symlinks outside the project I think. If we implemented this in the watcher we'd need to somehow crawl the whole file system to detect symlinks in the first place which would be very slow. We could potentially do it by detecting files that resolve to symlinks as we build the asset graph and keep nodes in the request graph to track watch roots. Then we'd have to re-watch all of those on startup and invalidate each of them for changes as opposed to just the project root.

(https://github.com/parcel-bundler/watcher/pull/32)

[ramirofages]

This is because the file watcher only watches for changes in the project root. It would be somewhat hard to implement support for symlinks outside the project I think. If we implemented this in the watcher we'd need to somehow crawl the whole file system to detect symlinks in the first place which would be very slow. We could potentially do it by detecting files that resolve to symlinks as we build the asset graph and keep nodes in the request graph to track watch roots. Then we'd have to re-watch all of those on startup and invalidate each of them for changes as opposed to just the project root.

(parcel-bundler/watcher#32)

This was actually working in parcel 1. Currently I had to revert back to parcel 1 while keeping its dependencies in parcel 2 to correctly build as module.

[krnlde]

This issue keeps me from migrating to v2, too. I wonder how other developers handle this workspaces issue - also considering monorepos are now en vogue. We can't possibly be alone here @ramirofages

[krnlde]

You guys should have a look at #6039 regarding the yarn.lock / package-lock.json files. This probably solves the issue for you too.

[kristojorg]

That linked issue doesn't resolve this for me. I don't have a dangling yarn.lock file, just the one in my primary repository I'm working on. Linked dependency changes don't trigger a recompile, and it seems even restarting parcel and running it with --no-cache don't trigger a full rebuild. I have the delete .parcel-cache in order to have the changes picked up : /

[krnlde]

Afaik parcel can't look outside its start context so if you want to watch all workspaces in a workspace root you'll need to start parcel in that root. Did you do that? Be extra sure to not have any lock files inside any of the workspaces. Parcel unfortunately has no warning for that and will just silently fall back to the workspace scope instead of root scope, which took a lot of lifetime from me in the past the point where I wanted to quit my job and start a lumber business :D

[kristojorg]

I am trying to link a dependency that is located in a separate folder and repository on my machine, but installed via npm. The node_modules/my-dep folder does change when I update the linked dependency, but parcel doesn't seem to pick up the changes there. Do you know if there is a way to have it watch and rebuild node_modules as well?

[krnlde]

As long as the node_modules is inside the parcel start context it works. Again parcel picks its start context by probing all folders for .lock files starting with the most-inner one from your start path: parcel serve workspaces/workspace-a/index.html probes:

1. workspaces/workspace-a/*.lock
2. workspaces/*.lock
3. *.lock

wherever a lock file is found it will not recognize changes beyond that folder. Ever. And they are not planning to change that.

[kristojorg]

Yep I hear you, but let me be more explicit about my setup. I have:

project-a/
    src/                      # code lives here
    example/
        index.html       # no lockfile here, no package.json, no node_modules
    package.json      # here is my script "npm run example": "parcel example/index.html"
    package-lock.json
    node_modules/
        project-b/
project-b/                # linked to ^
    src/
    dist/

It is the linked project-b dependency which is changing and not triggering a rebuild inside project-a when I run npm run example. So what I'm trying to say is that this isn't an issue of a misplaced or extra *.lock file it seems.

[krnlde]

So its linked by npm link and not by workspace logic? I couldn't repeat myself more but:

wherever a lock file is found it will not recognize changes beyond that folder

Edit: also to clarify, parcel doesn't watch the node_modules (since its essentially a black hole), it watches source files and triggers a reload when they change.

[mischnic]

If project-a contains a lockfile or a .git directory, it will be the project root and only files inside of that are watched.

Edit: also to clarify, parcel doesn't watch the node_modules (since its essentially a black hole), it watches source files and triggers a reload when they change.

No, it watches everything except for VCS directories and .parcel-cache: https://github.com/parcel-bundler/parcel/blob/ff6b7b4e97f95d8aabe3a93c22ec30f96b0fbf15/packages/core/core/src/RequestTracker.js#L918-L920

[kristojorg]

Okay yes it is the "Edit" there that is my problem then. The confusing thing is that this did work with parcel v1. When I would npm link package-b, parcel would rebuild package-a whenever it changed.

As mentioned, project-a does contain a lockfile and a .git directory, but node_modules are within that folder so I expected them to be watched, or at least not permanently cached when I restart with npm run example. I currently have to delete .parcel-cache in order for it to include the updated node_modules.

[krnlde]

Ah cool! Didn't know that. How about symlinks then?

[kristojorg]

@mischnic so it does appear that this is a legitimate bug and not intended behavior, then. Should node_modules possibly be excluded from being written to the .parcel-cache?

[mischnic]

How about symlinks then?

"only files inside of that are watched" essentially ignores symlinks. This is mostly because the file watching APIs from the various OSs only fire change events for the actual parent directories and don't traverse through every symlink

So node_modules is watched, but node_modules/symlinked-package/* isn't

So this is more of a technical limitation at the moment, as explained in https://github.com/parcel-bundler/parcel/issues/4332#issuecomment-706837311 and not a design decision

a legitimate bug and not intended behavior, then. Should node_modules possibly be excluded

What do you mean? It intentionally watches node_modules so that updating dependencies and rerunning Yarn triggers a Parcel rebuild for changed files.

[kristojorg]

I see, thanks for the explanation. I wonder if it would be possible to make an option to explicitly tell parcel about a symlinked dependency you would like to be watched, as this does seem like it would be a common use case.

What do you mean? It intentionally watches node_modules so that updating dependencies and rerunning Yarn triggers a Parcel rebuild for changed files.

I meant should node_modules be watched but not cached, such that when you restart parcel it is reading node_modules fresh and not using a cached version. I don't think I totally understand the value of including node_modules files in the cache if they are not bundled. Seems like I'm missing something tho : )

[padcom]

This is really bad... Parcel 1 used to work with symlinked packages as already mentioned in this thread. Not being able to symlink packages basically means one needs to restart the bundler every single time a change is made to external package.

Imagine working on an app that has a thin shell and a number of libraries that deliver parts of the app in the form of components. Unless you go monorepo it is just not possible to do it with Parcel right now.

That's really a shame. I wanted to base my next set of projects on Parcel 2, after waiting for so long for it to go GA and now it turns out it has this limitation :(

Any idea about other bundlers that don't have that limitation?

[JanMisker]

A very hacky workaround I found:

1. Run parcel with --no-cache, and
2. Whenever the locally linked packaged changes, save package.json to trigger a rebuild

But I definitely agree that this should just work, Create React App does pick up changes in packages that are added with npm link ../../another-project so that could be inspiration?

[xuqingkuang]

A very hacky workaround I found:

1. Run parcel with --no-cache, and
2. Whenever the locally linked packaged changes, save package.json to trigger a rebuild

But I definitely agree that this should just work, Create React App does pick up changes in packages that are added with npm link ../../another-project so that could be inspiration?

Good workaround, attach my code for example:

chokidar.watch(LINKED_DEPENDENCIES, {/* OPTIONS */}).on('all', () => {
    const projectPackagePath = path.resolve(PROJECT, 'package.json');
    const now = new Date();
    fs.utimes(projectPackagePath, now, now);
});

But there's a minor issue with the workaround, LINKED_DEPENDENCIES changed will trigger the whole page reload, not HMR.

[danmarshall]

@JanMisker thanks for the workaround idea.

2. Whenever the locally linked packaged changes, save package.json to trigger a rebuild

In my case, I have more than 2 levels of dependencies in a monorepo, and this technique only seems to work one level deep.

Parcel 1 "just worked" quite magically. Without this ability, Parcel 2 is a step back in productivity. I would not have updated if it weren't for the many security issues of Parcel1 's dependencies.

[nikoloza]

This is still an issue right?

I used to trigger reload (not hmr) by nodemon watching the symlinked repository and trigger package.json file change in the main one.

[gjermundgaraba]

I have the exact same issue. This works perfectly for other HMR systems I have worked with. I'm using Lerna to symlink in different components. In Vue for instance this works perfectly, but on a simple parcel page it doesn't catch up (And I have to reload with --no-cache all the time...). Parcel 2 is certainly not having a good DX story for this kind of setup.

[padcom]

Guys, is this still being considered if it will be changed? What's stopping you from fixing this problem?

[devongovett]

It's on the roadmap, but it will be a bit complex to implement. We somehow need to discover all of the watch roots as parcel builds a project for the first time. We also need to check them all on startup when building from cache.

In addition, we will have to figure out how to handle paths that are outside the project root. Parcel's cache is meant to be portable between machines, or if you move your project directory, but symlinks outside the project root break this.

Also, symlinks tend to break node_modules resolution. For example, if your project has a dependency on React, and you symlink another module which also has a dependency on React, you will get two copies of React in your build and it most likely won't work at runtime.

Speaking from personal experience in the past trying to symlink projects together, I would definitely not recommend a setup like this. It breaks many things, not just watching. Using yarn/npm workspaces is much better in basically every way.

[yume-chan]

Using yarn/npm workspaces is much better in basically every way.

yarn/npm workspaces are still symlinks:

Please note the fact that /workspace-a is aliased as /node_modules/workspace-a via a symlink.

https://classic.yarnpkg.com/lang/en/docs/workspaces/#toc-how-to-use-it

Automating the linking process as part of npm install and avoiding manually having to use npm link in order to add references to packages that should be symlinked into the current node_modules folder.

https://docs.npmjs.com/cli/v7/using-npm/workspaces

This is the problem: Parcel 2 now doesn't work with most monorepo solutions. (In my experience, it can detect file content changes in a Rush monorepo, but not file creation/deletion)

[devongovett]

To be clear, symlinks are fine as long as they point within the project root directory. That is the case with most monorepos, e.g. those managed by yarn or npm workspaces. I personally use these in several projects with parcel and it works fine.

[padcom]

Yeah, well unfortunately it all breaks down when you have modules in different repos and you use npm link which is the de facto standard way of linking libs in JS projects.

W dniu czw., 6.01.2022 o 05:57 Devon Govett @.***> napisał(a):

To be clear, symlinks are fine as long as they point within the project root directory. That is the case with most monorepos, e.g. those managed by yarn or npm workspaces. I personally use these in several projects with parcel and it works fine.

— Reply to this email directly, view it on GitHub https://github.com/parcel-bundler/parcel/issues/4332#issuecomment-1006285194, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEHICUFAHEL7UOJBZ7Y7HTUUUOMDANCNFSM4LKVYUSQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

[sangupta]

Another workaround based on fswatch that I have started using to trigger parcel 2 rebuilds:

$ fswatch -o /path/to/local/dependency/dist/library.js 
    | xargs -n1 -I{} 
        cp /path/to/local/dependency/dist/* /path/to/project/node_modules/mylib/dist

Library is declared as file dependency in my package.json:

"dependencies": {
    "react": "^17.0.2",
    "react-dom": "^17.0.2",
    "react-router-dom": "^5.3.0",
    "bedrock": "file:/Users/sangupta/git/mylibrary"
}

[ksjogo]

Does someone have an example with a working setup with npm workspaces? I tried setting up my new project with them, but still need to (re)start manually when referring to packages by name like:

import JSGDHost from '@kronbergerspiele/jsgdbridge'

Using a relative path works though

import JSGDHost from '../bridge'

[PhenX]

This issue is a true blocker, and I tend to regret having upgraded from parcel 1 because of this :( This is a PITA when working with workspaces now. Having builds that take 3 seconds instead of 4 is cool, but not when you need to stop, delete cache and rebuild.

I'm really hoping this issue (in the short term roadmap) is addressed quickly.

Also, thank you for this great bundler !

[karasma3]

I upgraded to Parcel 2 and i find it really inefficient to parcel build every time i do some changes due to parcel watch not working. In parcel 1 everything worked as expected. I am using this command parcel watch src/index.html Is there any news to this issue?

[padcom]

For the moment, I found a temporary solution when working with libraries. Instead of making it a build-time dependency, I have tried to use it as runtime dependency, using module federation. That way, the lib is a separate project that you deploy separately so the entire build process doesn't rely on linked node_modules

https://github.com/padcom/minimal-vue2-module-federation-example

I know this example doesn't use Parcel, but the idea is exactly the same.

[danmarshall]

In my case, I was able to make it work by moving the invocation of parcel to the project root. I also need to move my index.html page out of the packages/app directory to a folder off the root directory.

[devongovett]

If you are having problems, could you please explain your setup in more detail? Parcel 2 works fine with yarn workspaces for example. As long as the symlinks are within your project root (i.e. git repo), it should already work fine.

If you are symlinking outside, how do you set this up? Manual npm link or similar? Is this only temporary? Do you work with a team? How do you ensure all devs have the same setup? I feel that monorepos i.e. yarn/npm workspaces are the true solution here. Happy to eventually add support for watching outside the project root but there is complexity there and I want to understand why this issue gets so much attention. So, please help me understand your setups.

[PhenX]

Hello, @devongovett I made a repository to reproduce the issue I had : https://github.com/am-creations/parcel-issue-4332-symlink-watch

The project structure in this example is similar to our projects : /src/package.json and /vendor/whatever/package.json, so the "main" package.json file is not in the project root.

• I do work with a team
• The project uses internal libraries added via subtree or submodule
• The issue happens when applying changes to thes libraries
• All devs use NPM 7 / node 14 or 8 / node 16, the issue is the same
• We work on Windows

[devongovett]

Thanks. One question: why not place your package.json/package-lock.json in the root of your repo rather than inside src? That would solve this problem.

[PhenX]

That was an option I tried, but got issues as NPM has a inheritance mechanism that is a PITA because we use preinstall scripts : all modules in a directory inside the root execute the scripts, for example. And there are maybe issues I could not identify because I was blocked by the first issue. And as that it worked fine with parcel 1, we structured all our projects like this.

[devongovett]

Would an option to set the project root explicitly via the CLI to override the inferred root based on the lockfile work for you? Watching outside the project root is much harder, but if you could set the project root manually that might be a reasonable workaround?

[PhenX]

If you think it could work, I'm OK with this. Even further, do you think the common root directory could be found automatically? If not, could this option be set in the targets?

[bminer]

Just like @devongovett suggested, I was thinking about adding a CLI option that explicitly sets the project root, although I have an inexplicable inkling that this solution might be fraught with odd corner cases.

Perhaps a better approach is a CLI option that allows one to explicitly specify folders to watch outside of the project root. The project root would still be automatically determined as it is presently. Thoughts?

[devongovett]

The problem is it's not portable between machines. Every person on your team would need a different configuration potentially.

[bminer]

@devongovett - Agreed. Portability across machines could be compromised a bit if one could specify arbitrary folders to watch. Although playing devil's advocate, this at least allows users to do it. I can foresee a situation where users want to watch changes for a dependency that is npm link-ed into a project root. Many "hacks" like these are temporary while dependencies are still under heavy development.

Here's the most common use case, IMHO. Imagine, for example, a Git repository with backend, frontend, and common folders. The Parcel root project folder might be frontend (i.e. where package.json is located), but there might be isomorphic code shared between the backend and frontend inside of the common folder. In this case, it would be nice if Parcel could listen for changes in the common folder, even though this is a sibling folder outside of the project root.

[PhenX]

@devongovett @bminer as I mentionned earlier, do you think it's a viable option to find all symlinks (maybe with a max count), and watch the target folders of these simlinks, it would not require any change depending on the platform, developers, etc.

[devongovett]

You'd have to traverse potentially across the entire file system to find them, which sounds slow.

[theopolisme]

Perhaps an opt-in flag?

In our case, @bminer's description matches our setup as well. Across developer machines, the relative directory structure is consistent, though the absolute location of course would vary:

• ./monorepo <- pnpm workspace
  • ./apps
  • ./app-1 <- Parcel root
  • ./app-2
  • ./packages
  • ./package-1
  • ./package-2

The use-case is the same; developers make a change to package-1 and want it to live refresh app-1. Using pnpm, package-1 has been symlinked into app-1's node_modules. Right now, Parcel doesn't detect those changes.