search

vercel / next.js

The React Framework
https://nextjs.org
MIT License
124.88k stars 26.66k forks source link

operation not permitted #40760

Open Achaak opened 1 year ago

Achaak commented 1 year ago

Verify canary release

Provide environment information

Operating System:
  Platform: win32
  Arch: x64
  Version: Windows 10 Pro
Binaries:
  Node: 16.15.0
  npm: N/A
  Yarn: N/A
  pnpm: N/A
Relevant packages:
  next: 12.3.2-canary.0
  eslint-config-next: 12.3.1
  react: 18.2.0
  react-dom: 18.2.0

What browser are you using? (if relevant)

No response

How are you deploying your application? (if relevant)

No response

Describe the Bug

When I run the build I have this error

> Build error occurred
[Error: EPERM: operation not permitted, scandir 'C:\Users\axell\Documents\Development\pikas-template\apps\app\.next\standalone\apps\app\node_modules\next'] {
  errno: -4048,
  code: 'EPERM',
  syscall: 'scandir',
  path: 'C:\\Users\\axell\\Documents\\Development\\pikas-template\\apps\\app\\.next\\standalone\\apps\\app\\node_modules\\next'
}

And in my next.config.js I have this

require('dotenv-flow').config({ path: '../..' });

const withPWA = require('next-pwa');
const withPlugins = require('next-compose-plugins');
const withBundleAnalyzer = require('@next/bundle-analyzer')({
  enabled: process.env.ANALYZE === 'true',
});
const { join } = require('path');
const withTM = require('next-transpile-modules')([]);

const plugins = [withTM, withBundleAnalyzer, withPWA];

/**
 * @type {import('next').NextConfig}
 */
const nextConfig = {
  swcMinify: true,
  reactStrictMode: true,
  output: 'standalone',
  experimental: {
    outputFileTracingRoot: join(__dirname, '../../'),
  },
  i18n: {
    locales: ['en', 'fr'],
    defaultLocale: 'en',
  },
  pwa: {
    disable: process.env.NODE_ENV === 'development',
    dest: 'public',
  },
};

module.exports = withPlugins(plugins, nextConfig);

I found these issues that talk about it too:

36645

1537

Expected Behavior

I just want to build my app

Link to reproduction

https://github.com/Achaak/pikas-template

To Reproduce

Use this command

pnpm i

pnpm generate

pnpm build:packages

cd apps/app

pnpm build  <-- The bug appears here
balazsorban44 commented 1 year ago

Unfortunately, I cannot reproduce the issue myself, but it looks related to the output: "standalone" or outputFileTracingRoot settings.

To investigate further, here are some things we could try:

Achaak commented 1 year ago

After several tests with colleagues, the problem would be on windows.

And yes when I remove output: "standalone" ans outputFileTracingRoot its work but I need it

j2ghz commented 1 year ago

I'm hitting the same issue, pnpm + nextjs standalone + windows (but no outputFileTracingRoot)

> Build error occurred
[Error: EPERM: operation not permitted, mkdir '<snip>\.next\standalone\node_modules\next'] {
  errno: -4048,
  code: 'EPERM',
  syscall: 'mkdir',
  path: '<snip>\\.next\\standalone\\node_modules\\next'
}
info  - Collecting page data . ELIFECYCLE  Command failed with exit code 1.

This is what the folder .next\standalone\node_modules looks like: image

My guess is that this is caused by nextjs standalone trying to copy the symlinks/hardlinks/junctions pnpm creates instead of copying the content?

j2ghz commented 1 year ago

Unfortunately, I cannot reproduce the issue myself, but it looks related to the output: "standalone" or outputFileTracingRoot settings.

To investigate further, here are some things we could try:

  • Build with and without output: "standalone".

  • Check if it builds when outputFileTracingRoot is not utilized

  • Check a different package manager. yarn or npm, to see if the issue is pnpm-specific or not.

  • It would also be nice to see if we can pinpoint exactly which version of next this bug was introduced in. (A good candidate to start bisecting is v12.1.7-canary.48, when output: "standalone" was stabilized. If the issue reproduces still, the bug might have always existed.)

This is my minimal reproduction example, it's basically just 3 steps (powershell):

# Make dir if not exists
mkdir -Force "./next-40760-repro"
# cd to it
pushd "./next-40760-repro"
# delete any content from previous run
rm * -Recurse -Force 

# create-next-app with pnpm
pnpm create next-app --ts --use-pnpm .

# add output: 'standalone' to next.config.js
[regex]$pattern = ",\s+}"
$pattern.Replace((cat "next.config.js"), ",output: 'standalone',};", 1) | Set-Content "next.config.js"

# build
pnpm next build

popd
drewbitt commented 1 year ago

I can confirm this exact problem with Windows + pnpm. .next/standalone/node_modules includes dependencies in the .pnpm folder instead of the root node_modules folder, leading to Error: EPERM: operation not permitted, stat 'C:\GitHub\project\.next\standalone\node_modules\next'

It works fine with npm as dependencies are in root node_modules

Unsure if this stackoverflow issue is related, then it would mean this is an old issue https://stackoverflow.com/questions/70895922/pnpm-linking-with-standalone-nextjs-build

drewbitt commented 1 year ago

I have just tried it in Linux and it works perfectly. So is Windows + pnpm specific. Turbotrace on Windows does not fix the issue either. And building the standalone folder on Linux, which works, and moving it to Windows and running it still causes the error.

sandermaas commented 1 year ago

I encounter the same issue with this setup:

We use standalone output in combination with outputFileTracingRoot for monorepo usage.

If I run pnpm build with normal privileges I get a scandir error: errno: -4048, code: 'EPERM', syscall: 'scandir', Same command with admin privileges gives me the mkdir error: errno: -4048, code: 'EPERM', syscall: 'mkdir',

vixero-dev commented 1 year ago

I have the same problem with the following setup:

[Error: EPERM: operation not permitted, scandir '...\.next\standalone\node_modules\axios'] {
  type: 'Error',
  errno: -4048,
  code: 'EPERM',
  syscall: 'scandir',
  path: '...\\.next\\standalone\\node_modules\\axios'
}

For now, deleting .next folder for almost every build is the solution.

lveillard commented 10 months ago

This is still happening in 14.0.1

zfeher commented 10 months ago

I am also experiencing this issue:

Operating System:
  Platform: win32
  Arch: x64
  Version: Windows 10 Home
Binaries:
  Node: 18.17.1
  pnpm: 8.5.1
Relevant packages:
  next: 13.4.7
  eslint-config-next: 13.4.7
  react: 18.2.0
  react-dom: 18.2.0
  typescript: 5.1.6

It also happens with v13.5.6, v14.0.2 of Next.

Before the build removing only .next/standalone or even .next/standalone/node_modules works as a workaround.

Comparing the output of .next/standalone/node_modules with linux version seems similar, no actual directories for deps but links. Linux (WSL Ubuntu) build works though on second run. Could be the type of windows links wrong (symlink/hardlink/junction)? pnpm folder links looks different in Total Commander for example and they can be traversed, whereas standalone node_modules dir links are not traversable.

Playing a little with mklink command:

rem this one creates a file link which looks like a file and not traversable via Total Commander
mklink some-folder target-folder

rem both of these creates directory links which looks like how pnpm creates them and they are traversable
mklink /d some-folder target-folder2
mklink /j some-folder target-folder3

If the initial build can work with pnpm directory links (standalone node_modules are created, .pnpm has the real files) then it should work with directory links as well if output tracing could change the type of links it creates (dir => dir) somehow.

Hope this helps.

newbeea commented 10 months ago

same issue, "next": "14.0.2-canary.21" pnpm

mjad218 commented 9 months ago

The bug disappeared when I changed the dev script from 

    "dev": "next dev --turpo -p 3000 ",

to 

    "dev": "next dev -p 3000 ",
AhmedBaset commented 7 months ago

In my case, this issue occurs on output: "standalone". https://github.com/vercel/next.js/blob/4125069840ca98981f0e7796f55265af04f3e903/packages/next/src/build/utils.ts#L1957 It seems like fs.symlink is not supported ideally at Windows (at least at mine).

I fixed it by enabling Windows Developer Mode

tsafadi commented 5 months ago

Same error for me.

ldv-orbyta commented 5 months ago

same here!

Edit by maintainer bot: Comment was automatically minimized because it was considered unhelpful. (If you think this was by mistake, let us know). Please only comment if it adds context to the issue. If you want to express that you have the same problem, use the upvote 👍 on the issue description or subscribe to the issue for updates. Thanks!

dallyh commented 5 months ago

In my case, this issue occurs on output: "standalone".

https://github.com/vercel/next.js/blob/4125069840ca98981f0e7796f55265af04f3e903/packages/next/src/build/utils.ts#L1957

It seems like fs.symlink is not supported ideally at Windows (at least at mine). I fixed it by enabling Windows Developer Mode

Dropping this here: https://superuser.com/questions/104845/permission-to-make-symbolic-links-in-windows-7 also helps.

nibblesnbits commented 4 months ago

Seeing this seemingly at random today.

❯ npm run build -- --debug

> unad-web@0.1.0 build
> next build

 ⚠ You are using a non-standard "NODE_ENV" value in your environment. This creates inconsistencies in the project and is strongly advised against. Read more: https://nextjs.org/docs/messages/non-standard-node-env
   ▲ Next.js 14.1.0
   - Environments: .env

   Skipping linting
 ✓ Checking validity of types
   Creating an optimized production build ...
   Using tsconfig file: ./tsconfig.build.json
glob error [Error: EPERM: operation not permitted, scandir 'C:\Users\sgcol\Cookies'] {
  errno: -4048,
  code: 'EPERM',
  syscall: 'scandir',
  path: 'C:\\Users\\sgcol\\Cookies'
}
Failed to compile.

Error: EPERM: operation not permitted, scandir 'C:\Users\sgcol\Cookies'

> Build failed because of webpack errors
npm ERR! Lifecycle script `build` failed with error:
npm ERR! Error: command failed
npm ERR!   in workspace: unad-web@0.1.0
npm ERR!   at location: D:\The-UnAd\monov2\web\unad-web

It also sometimes complains about the path `'C:\Users\sgcol\Application Data'.

No idea how to even begin with this.

AshleyJackson commented 4 months ago

In my case, this issue occurs on output: "standalone".

https://github.com/vercel/next.js/blob/4125069840ca98981f0e7796f55265af04f3e903/packages/next/src/build/utils.ts#L1957

It seems like fs.symlink is not supported ideally at Windows (at least at mine). I fixed it by enabling Windows Developer Mode

Enabling Developer 100% fixed this issue for me. I was seeing this issue in PNPM and Bun. Enabled this setting and it allowed be to build without any issues straight away.

Thanks @AhmedBaset

zrthomas commented 4 months ago

Having the same problem on windows with pnpm during build. output: "standalone" is not enabled. Windows Developer Mode does not fix the issue.

nibblesnbits commented 4 months ago

What's even more strange is that it's failing with different directories every time, but always in my user directory.

glob error [Error: EPERM: operation not permitted, scandir 'C:\Users\sgcol\Cookies'] {

glob error [Error: EPERM: operation not permitted, scandir 'C:\Users\sgcol\Application Data'] {

What could anything in the build possibly be doing in my user directory??? My code is in a completely separate drive.

If @AshleyJackson is right, this means the failure is related to trace files. Isn't that data you guys phone home with for metrics/etc?

After watching my outgoing traffic, it's definitely happening when the build phones home to telemetry.nextjs.org.

........ Are you guys phoning home data from my user directory???

AshleyJackson commented 4 months ago

@nibblesnbits There does seem to be some telemetry at a quick glance, haven't put any effoirt in to see what it does though. Have fun. https://github.com/vercel/next.js/blob/3d143791c640f57632490ed229212dcfc87382c1/packages/next/src/telemetry/storage.ts#L313

nibblesnbits commented 4 months ago

Well after just running Process Monitor I can see that the process spawned by the build attempts to read my entire user directory. Every file. I'll keep digging.

AshleyJackson commented 4 months ago

@nibblesnbits You could run fiddler to try and intercept the data it's pushing. You might need to disable certificate validation in the nodejs config.

nibblesnbits commented 4 months ago

I'm setting up mitmproxy already 😂

zrthomas commented 4 months ago

This error occurs for me even with telemetry disabled when building locally. Project deploys just fine on vercel. glob error [Error: EPERM: operation not permitted, scandir 'C:\Users\zthom\Application Data'] { errno: -4048, code: 'EPERM', syscall: 'scandir', path: 'C:\\Users\\zthom\\Application Data' }

nibblesnbits commented 4 months ago

Can anyone give some guidance on narrowing down where this bug might be? I'd love to submit a PR, but I can't seem to find where the glob in question is even defined.

nibblesnbits commented 4 months ago

This error occurs for me even with telemetry disabled when building locally. Project deploys just fine on vercel. glob error [Error: EPERM: operation not permitted, scandir 'C:\Users\zthom\Application Data'] { errno: -4048, code: 'EPERM', syscall: 'scandir', path: 'C:\\Users\\zthom\\Application Data' }

Yeah, that's the thing. I feel like the fix is to find the glob pattern that's for some reason looking for too much. At least on Windows.

Cybermb commented 4 months ago

Jumping on the "same issue" train. To workaround this issue set outputFileTracing: false in next config. Also, for me, it seems that this issue is not happening on different windows machine with same project code. It could be something related to windows configuration

nibblesnbits commented 4 months ago

Jumping on the "same issue" train. To workaround this issue set outputFileTracing: false in next config.

It works!!!!!!!!!

Thank you!. Go figure it takes someone NOT working at Vercel to provide a workaround that actually works.

Cybermb commented 3 months ago

@nibblesnbits Do you in any chance have monorepo setup or referencing a local/workspace package in your code?

I am trying to narrow down the issue and it seems that referencing a workspace package in monorepo setup causes build fail with operation not permitted, scandir. Removing all the code that imports said package fixes this issue.

nibblesnbits commented 3 months ago

@Cybermb, I do, actually. There is a package.json at the root, and it references three workspaces, one in a ./web directory (the next project), and two more in a ./packages directory. My next project references both packages.

Cybermb commented 3 months ago

Seems I have managed to find the solution (for me at least). I have prisma database as a shared package in monorepo. The root cause was having output = "../src/generated/client" which generated prisma client outside of "node_modules" directory. For some reason, nextjs compiler did not like that. I had to specify the output directory to output = "../node_modules/.prisma/client" to solve this (you can just remove output option entirely if you have single prisma schema in whole project). The rest was just adjust paths/aliases in tsconfig.json files.

For people with same issue, if above does not work, you can try one of following:

  1. Assuming you are using Windows, move project to main drive (C:)
  2. Set outputFileTracing: false in your next config file. This option, however, will be removed in NextJS 15
nibblesnbits commented 3 months ago

Right that's the problem. There needs to be a permanent solution. Not something that will break again 6 months from now.

From: Mantas @.> Sent: Saturday, May 18, 2024 4:53:27 PM To: vercel/next.js @.> Cc: Stephen Collins @.>; Mention @.> Subject: Re: [vercel/next.js] operation not permitted (Issue #40760)

Seems I have managed to find the solution (for me at least). I have prisma database as a shared package in monorepo. The root cause was having output = "../src/generated/client" which generated prisma client outside of "node_modules" directory. For some reason, nextjs compiler did not like that. I had to specify the output directory to output = "../node_modules/.prisma/client" to solve this (you can just remove output option entirely if you have single prisma schema in whole project). The rest was just adjust paths/aliases in tsconfig.json files.

For people with same issue, if above does not work, you can try one of following:

  1. Assuming you are using Windows, move project to main drive (C:)
  2. Set outputFileTracing: false in your next config file. This option, however, will be removed in NextJS 15

— Reply to this email directly, view it on GitHubhttps://github.com/vercel/next.js/issues/40760#issuecomment-2119000342, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABODKJIK2734ZQNJ5RTJPFTZC65UPAVCNFSM6AAAAAAQSGWOY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJZGAYDAMZUGI. You are receiving this because you were mentioned.Message ID: @.***>

ivan2214 commented 3 months ago

You need to enable Windows Developer Mode so that Preconstruct can create symlinks - https://www.howtogeek.com/292914/what-is-developer-mode-in-windows-10/

This works for me 🫡

asheemak commented 2 months ago

for me running the "pnpm run build" in admin mode fix it

rubenmamo commented 1 month ago

You need to enable Windows Developer Mode so that Preconstruct can create symlinks - https://www.howtogeek.com/292914/what-is-developer-mode-in-windows-10/

This works for me 🫡

I can confirm that this was resolved for me when I turned on developer mode in Windows 11.