[turborepo] Workspace packages not found if located behind symlink

[blorgon1]

What version of Turborepo are you using?

1.6.2

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Linux

Describe the Bug

When the path to a workspace package contains a symlink, turborepo can't find the package. I am using submodules for some of my workspace packages, and would like to keep all submodules in the same directory, while still linking them into the desired locations.

My repo is set up like this:

apps/
  app-a/
    package.json
      |-> widget-a@*
widgets/
  widget-a -> ../submodules/widget-a
submodules/
  widget-a/
    package.json

With a pnpm-workspace.yaml like this:

packages:
  - "apps/*"
  - "widgets/*"

Then when building app-a, this error is encountered:

Cannot find module 'widget-a' or its corresponding type declarations.

Without using the symlink (placing the submodule in the widgets directory), the build passes as expected.

This behaviour differs from pnpm, as pnpm is still able to locate the workspace packages behind a symlink:

> pnpm ls --depth -1 -r
/project/dir

app-a@1.0.0 /project/dir/apps/app-a

widget-a@1.0.0 /project/dir/widgets/widget-a

Expected Behavior

Packages should be found when they are located behind symlinks. The behaviour should be identical to when the package is in place of the symlink.

To Reproduce

Move a workspace package to a different location (within the same project folder, just not in a workspace package path), and place a symlink to it in the original location.

Reproduction Repo

https://github.com/blorgon1/turbo-symlink-repro

[chris-olszewski]

Hi @blorgon1, could you provide a reproduction repo? Module resolution is a responsibility of the package manager so it's odd that pnpm is able to find the package in ls but not whatever command you have set for building app-a.

[blorgon1]

Hi @chris-olszewski, thanks for your response.

Here is a reproduction repo: https://github.com/blorgon1/turbo-symlink-repro

[chris-olszewski]

Thanks for the repro! It looks like our globbing library isn't following symlinks. I should be able to get a fix up soon.

[nathanhammond]

@chris-olszewski This was an intentional change introduced in https://github.com/vercel/turbo/pull/1275/commits/494b89423c189781d0d0a5fae5382bfcdd3b300c

We need to make sure that we don't regress #1265.

[blorgon1]

Ah okay, that's unfortunate given that using a folder for submodules is a common practice.

From the fix commit message:

This caused us to copy their pointed-to contents as if the symlink was a directory, and then when copying the link itself, throw an error because it already existed (as a directory...).

It seems that an alternate solution would be to skip copying symlinks but still traverse them?

Anyways it's easy to work around, thanks for looking into it!

[pc-erin]

I'm having almost the same issue, but in my case it's caused by the symlink that npm creates to connect workspace dependencies. Do we need to manually copy and workspace dependencies to the output image in our dockerfile? This seems like the kind of default behavior that turbo repo should handle.

[andymac4182]

This is causing lots of issues for us :( I understand the reason for disabling symlinks but please at least give us an option to enable them again if we have a need to.

[moekify]

@chris-olszewski is this still happening? :)