Open rhlsthrm opened 2 years ago
Hey @rhlsthrm,
That error usually occurs because either the package @connext/nxtp-router
is either not listed in that bundled package's dependencies
.
Looking at the package.json
, I can't see it listed. https://github.com/connext/nxtp/blob/amarok/packages/agents/sequencer/package.json#L47
I added the following to dependencies
"@connext/nxtp-router": "workspace:*",
"@connext/nxtp-txservice": "workspace:*",
"@connext/nxtp-utils": "workspace:*",
"@connext/nxtp-subgraph": "workspace:*",
And then it bundled.
If you're still experiencing issues with it, please re-open this issue.
@ojkelly but those packages are not supposed to be part of the sequencer
package's dependencies. The sequencer
only uses a subset of the packages.
My assumption is that the point of this is only to use the workspace packages where this is a dependency.
@rhlsthrm fair enough, something else must be at play then.
I just checked .yarnrc.yml
and the project is set to nodeLinker: node-modules
. While that doesn't strictly block yarn bundle
, it's less predictable than nodeLinker: pnp
.
Bundling works after switching to pnp
linking. I don't know if there's other reason's you can't use pnp
, so it may not be the option here. Though, for bundling I do recommend using pnp
if you can.
Let's re-open to see if we can find the cause in this case.
We have not tried pnp yet. I'm worried about breaking some other downstream workflow using pnp but I will try it and see if everything else works.
Hey @ojkelly,
I've encountered this same issue with trying to bundle a package that exists within a monorepo with nodeLinker
set to node-modules
in .yarnrc.yml
. We have a React Native project in our monorepo, which is why we've avoided using pnp
.
I've tried excluding the React Native project via --exclude
and a .bundleignore
file but it produces the same error, even though we have no dependencies on our react-native project.
Error output:
❯ yarn bundle --exclude "@project/mobile" --ignore-file .bundleignore
➤ YB1000: ┌ Prepare /Users/.../project/apps/server for bundling
➤ YB1000: │ Preparing temporary directory
➤ YB1000: │ Copying repo to temporary directory
Internal Error: ENOENT: no such file or directory, chmod '/private/var/folders/38/63ftm4nj1_b89c4r9crxm39h0000gn/T/xfs-ce8f7901/apps/mobile/ios/Pods/Headers/Private/RNFBApp/RCTConvert+FIRApp.h'
Does --exclude
explicitly exclude those packages or file paths from being copied to the temp directory?
Nothing prevents copying all the files at the moment, as we're using an os file copy to setup the tmp dir.
.bundleignore
prevents those files from hanging around in the final zip or in the tmp folder (usually done for docker).
It sounds like you might be onto the right solution. If you're interested in making a PR the relevant area is here https://github.com/ojkelly/yarn.build/blob/main/packages/plugins/plugin-bundle/src/commands/bundle/index.ts#L355
Otherwise I think I can get to solving this in the next few weeks.
I think an ideal approach is to take the --exclude
folders, and subtract them from a list of files to copy instead of taking a full copy of the whole repo.
Hi :) any updates on that?
Unfortunately, I came across the same issue ... I tried multiple recipes ...also as you already mentioned the nodeLinker: pnp
works except that I can not use PnP (which is amazing) I must stick with node-modules as I'm using prisma in my project, and I'm pretty sure many fellas out there are using Prisma ...
In the meantime, I was looking at the stack trace I've found that it usually happens within the removeEmptyDirectories
function.
Unfortunately, I came across the same issue ... I tried multiple recipes ...also as you already mentioned the
nodeLinker: pnp
works except that I can not use PnP (which is amazing) I must stick with node-modules as I'm using prisma in my project, and I'm pretty sure many fellas out there are using Prisma ... In the meantime, I was looking at the stack trace I've found that it usually happens within theremoveEmptyDirectories
function.
I think what we probably need to do here to get this unblocked is add error handling around that area to catch, and probably ignore the errors thrown.
The proper fix sounds like ensuring we check if a file/folder exists before deleting it.
then again, if we try to delete and get ENOENT: no such file or directory
it’s basically the same.
My only concern, again it's your code, that if you do that it might break (or not) the integrity of the operation: why is it attempting to delete something that doesn't exist, in my case I think it was a symbolic link pointing to my workspace packages.
My only concern, again it's your code, that if you do that it might break (or not) the integrity of the operation:
Yep fair. In this case the only point of that function removeEmptyDirectories
is to help the [removeExcluded](https://github.com/ojkelly/yarn.build/blob/f0bace2ef13d0e36cf54a739c43db3b76cd4fcf8/packages/plugins/plugin-bundle/src/commands/bundle/index.ts#L257)
in reducing the size of the bundle.
This error appeared around this code in the past https://github.com/ojkelly/yarn.build/pull/120 which fixed #119.
On Windows, when traversing directories to delete empty directories, the bundle command incorrectly joined paths with
path.join
whenppath.join
should have been used. As a result, theremoveEmptyDirectories
recursed into invalid paths and the check for whether the invalid path is a directory failed due toENOENT: no such file or directory
.
It's preferable for the bundle to be marginally bigger, than break in some circumstances.
why is it attempting to delete something that doesn't exist
I think #142 is likely related to this, because it appears we've followed a path, and tried to delete something that does not exist.
If you're able to provide a link to a repro I think I know where we need to look now. Otherwise, if you have a bit of time, I'd drop some console.log
on everything in https://github.com/ojkelly/yarn.build/blob/f0bace2ef13d0e36cf54a739c43db3b76cd4fcf8/packages/plugins/plugin-bundle/src/commands/bundle/index.ts#L228.
And see what it's getting tripped up on.
Again, that functions job is just to reduce the bundle size.
Also see there's two uses of removeExcluded
, the first is for each package which doesn't clear empty dirs, and the second is for the global excluded list which does clear empty dirs https://github.com/ojkelly/yarn.build/blob/f0bace2ef13d0e36cf54a739c43db3b76cd4fcf8/packages/plugins/plugin-bundle/src/commands/bundle/index.ts#L542
in my case I think it was a symbolic link pointing to my workspace packages.
Can you expand on this? is it just like the packages/**/package/node_modules
or something in the top node_modules
?
We run yarn install before reducing the bundle size to let yarn itself remove any unused dependencies (of which there may be some, as we have removed your other local packages that are needed).
I feel like we're finally getting closer to resolving this bug.
Was any progress ever made on this? I’m running into the same issue.
This also happens if you have python code in your project. In our case, it's a top-level "jobs" folder which is not connected at all to yarn/node/js, but gets copied just by being in the same repo.
➤ YB1000: │ Preparing temporary directory
➤ YB1000: │ Copying repo to temporary directory
Internal Error: EPERM: operation not permitted, chmod '/private/var/folders/8f/t4ltlkgs3p7d5g0s9p_j_k_h0000gn/T/xfs-b41fd866/jobs/annotation-history/venv/bin/python3'
Error: EPERM: operation not permitted, chmod '/private/var/folders/8f/t4ltlkgs3p7d5g0s9p_j_k_h0000gn/T/xfs-b41fd866/jobs/annotation-history/venv/bin/python3'
(venv/bin/python3
is a link to /usr/bin/python3
)
I was really surprised that yarn bundle
has to copy everything at all. For large projects, I imagine it'd be even more of an issue.
Describe the bug I have a yarn workspace, I am trying to use this plugin to bundle and create docker images. When I try to bundle a package to be dockerized, I get an error:
To Reproduce Steps to reproduce the behavior:
amarok
: https://github.com/connext/nxtp/tree/amarokyarn bundle
inpackages/agents/sequencer
Expected behavior Bundle happens.
Screenshots
Desktop (please complete the following information):
Additional context Yarn version:
3.2.0