vercel / next.js

The React Framework
https://nextjs.org
MIT License
124.73k stars 26.62k forks source link

vercel error no such file or directory, open '/var/task/node_modules/shiki/themes/one-dark-pro.json' #52711

Open 2820402 opened 1 year ago

2820402 commented 1 year ago

Verify canary release

Provide environment information

Operating System:
      Platform: darwin
      Arch: arm64
      Version: Darwin Kernel Version 22.5.0: Thu Jun  8 22:22:19 PDT 2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8103
    Binaries:
      Node: 16.16.0
      npm: 9.8.0
      Yarn: 1.22.19
      pnpm: 7.16.0
    Relevant Packages:
      next: 13.4.10
      eslint-config-next: 13.4.10
      react: 18.2.0
      react-dom: 18.2.0
      typescript: 5.1.6
    Next.js Config:
      output: N/A

Which area(s) of Next.js are affected? (leave empty if unsure)

App Router

Link to the code that reproduces this issue or a replay of the bug

.

To Reproduce

import { serialize } from 'next-mdx-remote/serialize';
import readingTime from 'reading-time';
import remarkGfm from 'remark-gfm';
import rehypeSlug from 'rehype-slug';
import rehypeAutolinkHeadings from 'rehype-autolink-headings';
import rehypePrettyCode from 'rehype-pretty-code';

/** @type {import('rehype-pretty-code').Options} */
const options = {
    theme: 'one-dark-pro',
    onVisitLine(node: any) {
        if (node.children.length === 0) {
            node.children = [{ type: 'text', value: ' ' }];
        }
    },
    onVisitHighlightedLine(node: any) {
        node.properties.className.push('line--highlighted');
    },
    onVisitHighlightedWord(node: any) {
        node.properties.className = ['word--highlighted'];
    },
};

export async function mdxToHtml(source: any) {
    const mdxSource = await serialize(source, {
        mdxOptions: {
            remarkPlugins: [remarkGfm],
            rehypePlugins: [
                rehypeSlug,
                [rehypePrettyCode, options],
                [
                    rehypeAutolinkHeadings,
                    {
                        properties: {
                            className: ['anchor'],
                        },
                    },
                ],
            ],
            format: 'mdx',
        },
    });

    return {
        html: mdxSource,
        wordCount: source.split(/\s+/gu).length,
        readingTime: readingTime(source).text,
    };
}

got this error in vercel logs while building pages and i am using app router

ENOENT: no such file or directory, open '/var/task/node_modules/shiki/themes/one-dark-pro.json'
and i also install shiki package

Describe the Bug

got this error in vercel logs while building pages and i am using app router

Expected Behavior

mdx

Which browser are you using? (if relevant)

brave

How are you deploying your application? (if relevant)

Vercel

Complexia commented 1 year ago

Hey, I have the same issue. My json file is located in root, and it works fine locally, but when trying to load it on Vercel, I get the ENOENT: no such file or directory, open '/var/task/file.json error.

Using NextJS 13 app router

ghosthash commented 1 year ago

Same issue here.

2820402 commented 1 year ago

Hey, I have the same issue. My json file is located in root, and it works fine locally, but when trying to load it on Vercel, I get the ENOENT: no such file or directory, open '/var/task/file.json error.

Using NextJS 13 app router

I am also using the app directory did you find some fix?

shuding commented 1 year ago

This is likely related to next-mdx-remote and Shiki. Sounds like the file tracking is not handling it correctly.

2820402 commented 1 year ago

This is likely related to next-mdx-remote and Shiki. Sounds like the file tracking is not handling it correctly.

did you find some fix for that?

edgarasben commented 1 year ago

Having the same issue

kacperadler commented 1 year ago

Same here :/

2820402 commented 1 year ago

Still no response from nextjs team😔

Lu-TheCoder commented 1 year ago

I am having the same issue unfortunately

L0giXX commented 1 year ago

Did anyone find a solution to this problem?

Lu-TheCoder commented 1 year ago

Did anyone find a solution to this problem?

does it occur when you navigate to a dynamic page? for example if your building a blog website it would be at www.website.com/blog/your-dynamic-blog-post

if so then you might need to use next.js 13 generateStaticParams() function if your not already doing so, as this occurred with me as well until I generated my static params and the error went away

2820402 commented 1 year ago

Did anyone find a solution to this problem?

does it occur when you navigate to a dynamic page? for example if your building a blog website it would be at www.website.com/blog/your-dynamic-blog-post

if so then you might need to use next.js 13 generateStaticParams() function if your not already doing so, as this occurred with me as well until I generated my static params and the error went away

I am using generatestaticparamas in my dynamic route but I set a limit for 100 blogs to build on build time as if I build all the blogs at build time this take time so I will then generate the rest of blogs on demand when someone come to that slug I will then call the cms to check if there is any blog related to that slug if not i return 404 this is more efficient way

alivault commented 1 year ago

Did anyone find a solution to this problem?

I found the answer here: https://github.com/shikijs/shiki/issues/138#issuecomment-1057471160

basically in nextjs there are 3 states:

We want Shiki on getServerSideProps. This means:

The trick is:

The key here is to let Vercel nft to know about the existence of Shiki/themes and shiki/languages so they are included in the production run-time

Sample code: https://github.com/thien-do/memos.pub/blob/a3babb1f149f05c43012278331f885d81f5fcfac/lib/mdx/plugins/code.ts

credit to thien-do

dm1nh commented 1 year ago

Did anyone find a solution to this problem?

I found the answer here: shikijs/shiki#138 (comment)

basically in nextjs there are 3 states:

* build-time, on server -> this is getStaticProps

* run-time, on server -> this is getServerSideProps, or getStaticProps in case of ISR, which is this issue

* run-time, on client

We want Shiki on getServerSideProps. This means:

* ALL languages are available

* ALL themes are available

* NONE of them are bundled into client side

The trick is:

* Copy shiki/themes and shiki/languages to somewhere outside of node_modules, maybe under lib/shiki

* Touch these folders in a server side function (e.g. fs.readdirSync)

* Done!

The key here is to let Vercel nft to know about the existence of Shiki/themes and shiki/languages so they are included in the production run-time

Sample code: https://github.com/thien-do/memos.pub/blob/a3babb1f149f05c43012278331f885d81f5fcfac/lib/mdx/plugins/code.ts

credit to thien-do

Not work for me in a monorepo setup with Turborepo. I followed this and when I check the build output (.next directory), shiki assets are not still there. My app runs perfectly in local machine.

alivault commented 1 year ago

@dangminhngo you won't see the shiki assets in the .next folder because Next.js uses webpack to bundle and optimize the application. During this process, files might be renamed, minified, and split into different chunks. This could include combining various dependencies into common chunks, so finding individual files might be challenging.

The critical aspect of the solution above is that it makes Vercel aware of the existence of the Shiki assets so they are included at runtime on the server. The exact location in the build folder might not be straightforward to identify, but as long as the deployed site is working as intended, it indicates that the assets are being correctly handled.

clearly-outsane commented 10 months ago

I did that ^ but now I just get the same error with the new path of /lib/shiki. Is it supposed to be a different path for monorepos with a project that uses app directory?

mdavish commented 10 months ago

Any updates from the Vercel team here @shuding ? Feels like this is something that should just work without having to hack NFT

remarkablejames commented 10 months ago

No fix for this issue yet? I am having the same problem in production (Vercel)

jofelipe commented 9 months ago

Facing the same issue in production (Vercel), but my build has no errors Tried a lot of changes in my files directory but it didn't work :(

jofelipe commented 9 months ago

The trick is:

  • Copy shiki/themes and shiki/languages to somewhere outside of node_modules, maybe under lib/shiki
  • Touch these folders in a server side function (e.g. fs.readdirSync)
  • Done!

The key here is to let Vercel nft to know about the existence of Shiki/themes and shiki/languages so they are included in the production run-time

Sample code: https://github.com/thien-do/memos.pub/blob/a3babb1f149f05c43012278331f885d81f5fcfac/lib/mdx/plugins/code.ts

credit to thien-do

Tricky hack but works!

gokulkrishh commented 9 months ago

Did anyone find a solution to this problem?

I found the answer here: shikijs/shiki#138 (comment)

basically in nextjs there are 3 states:

  • build-time, on server -> this is getStaticProps
  • run-time, on server -> this is getServerSideProps, or getStaticProps in case of ISR, which is this issue
  • run-time, on client

We want Shiki on getServerSideProps. This means:

  • ALL languages are available
  • ALL themes are available
  • NONE of them are bundled into client side

The trick is:

  • Copy shiki/themes and shiki/languages to somewhere outside of node_modules, maybe under lib/shiki
  • Touch these folders in a server side function (e.g. fs.readdirSync)
  • Done!

The key here is to let Vercel nft to know about the existence of Shiki/themes and shiki/languages so they are included in the production run-time

Sample code: https://github.com/thien-do/memos.pub/blob/a3babb1f149f05c43012278331f885d81f5fcfac/lib/mdx/plugins/code.ts

credit to thien-do

This worked for me. Thanks.

smohadjer commented 8 months ago

I have the same problem and I'm not using any frameworks even. My json folder is in root and it works with vercel dev locally, but when deployed the path is logged as /var/task/json/data.json and can't be found, though I see on Vercel's website under source > output that the json folder and data.json file exist. In my serverless function I'm reading the file like this:

fs.readFileSync(path.join(process.cwd(), 'json/data.json'));
smohadjer commented 8 months ago

Hey, I have the same issue. My json file is located in root, and it works fine locally, but when trying to load it on Vercel, I get the ENOENT: no such file or directory, open '/var/task/file.json error.

Using NextJS 13 app router

I had the same problem and could fix it by moving my json file to public folder. So in my project root I have api and public folders and in my api serverless function I can read json like this:

import path from 'path';
import fs from 'fs';

const jsonPath = path.join(process.cwd(), 'public', 'data.json');
const jsonContent = fs.readFileSync(jsonPath, 'utf8');
helloanoop commented 7 months ago

My issue was related to shiki. This https://github.com/vercel/next.js/issues/52711#issuecomment-1674254848 helped me 🙏

Commit ref: https://github.com/brulang/bru-website/commit/2a85019e217c0ee6621cdf0707a06425649f965a

linzhe141 commented 5 months ago

Having the same issue

linzhe141 commented 5 months ago

Having the same issue

I feel like it is caused by some caching strategy. If I use the latest hash URL of vercel, it will work normally.

ctessier commented 3 months ago

const jsonPath = path.join(process.cwd(), 'public', 'data.json');

I have my file in the public folder too, and do exactly what you do here, but I still get the following error:

Error: ENOENT: no such file or directory, open '/var/task/public/images/map_western_cape_kruger.jpg'

The picture is available when browsing https://project-name.vercel.app/images/map_western_cape_kruger.jpg

Any idea? Thank you

linzhe141 commented 3 months ago

const jsonPath = path.join(process.cwd(), 'public', 'data.json');

I have my file in the public folder too, and do exactly what you do here, but I still get the following error:

Error: ENOENT: no such file or directory, open '/var/task/public/images/map_western_cape_kruger.jpg'

The picture is available when browsing https://project-name.vercel.app/images/map_western_cape_kruger.jpg

Any idea? Thank you

I changed the domain and it worked

image

ctessier commented 3 months ago

I changed the domain and it worked

I don't understand. My issue is the error on the backend when trying to load a page which has a server logic of reading the image. I don't see the relation with the domain, as it is only backend issue here.

linzhe141 commented 3 months ago

I changed the domain and it worked

I don't understand. My issue is the error on the backend when trying to load a page which has a server logic of reading the image. I don't see the relation with the domain, as it is only backend issue here.

I also use fs to read files in rsc, and this problem occurred, But when I change the domain, it works

linzhe141 commented 3 months ago

const jsonPath = path.join(process.cwd(), 'public', 'data.json');

I have my file in the public folder too, and do exactly what you do here, but I still get the following error:

Error: ENOENT: no such file or directory, open '/var/task/public/images/map_western_cape_kruger.jpg'

The picture is available when browsing https://project-name.vercel.app/images/map_western_cape_kruger.jpg

Any idea? Thank you

when I use this hash URL, it works. image

ctessier commented 3 months ago

when I use this hash URL, it works.

You don't get it. The image is accessible on the different domains as it is in the public folder (branch, preview, etc.). The problem is when trying to reading it from the server with fs.readFileSync for instance. On the frontend I get this:

image

In the logs I have: Error: ENOENT: no such file or directory, open '/var/task/public/images/map_western_cape_kruger.jpg'

Here is my code:

import path from 'path';
import getImageSize from 'image-size';
import Image, { type ImageProps } from 'next/image';
import { cn } from '@/libs/utils';

interface MdxImageProps extends React.ImgHTMLAttributes<HTMLImageElement> {}

export default function MdxImage(props: MdxImageProps) {
  if (!props.src) return;

  const newProps = { ...props };
  const isLocalImage = !props.src.startsWith('http');

  newProps.className = cn('m-auto', newProps.className);

  if (!props.width && !props.height && isLocalImage) {
    const filePath = path.join(process.cwd(), 'public', props.src);
    const dimensions = getImageSize(filePath); // <===== I guess it breaks here

    newProps.width = dimensions.width;
    newProps.height = dimensions.height;
  }

  return <Image {...(newProps as ImageProps)} alt={newProps.alt || ''} />;
}
Zen-cronic commented 2 months ago

const jsonPath = path.join(process.cwd(), 'public', 'data.json');

I have my file in the public folder too, and do exactly what you do here, but I still get the following error:

Error: ENOENT: no such file or directory, open '/var/task/public/images/map_western_cape_kruger.jpg'

The picture is available when browsing https://project-name.vercel.app/images/map_western_cape_kruger.jpg

Any idea? Thank you

I'm facing the same issue. Already tried these to no avail:

  1. placing my content (.md) under the public folder
  2. using path.join(process.cwd(), "public", ...) in the serverless function
  3. using generateStaticParams() for each piece of content (e.g., blogposts)
rakhaviantoni commented 1 month ago

const jsonPath = path.join(process.cwd(), 'public', 'data.json');

I have my file in the public folder too, and do exactly what you do here, but I still get the following error:

Error: ENOENT: no such file or directory, open '/var/task/public/images/map_western_cape_kruger.jpg'

The picture is available when browsing https://project-name.vercel.app/images/map_western_cape_kruger.jpg

Any idea? Thank you

Any update to solve this? Facing the same issue here

rakhaviantoni commented 1 month ago

const jsonPath = path.join(process.cwd(), 'public', 'data.json');

I have my file in the public folder too, and do exactly what you do here, but I still get the following error: Error: ENOENT: no such file or directory, open '/var/task/public/images/map_western_cape_kruger.jpg' The picture is available when browsing https://project-name.vercel.app/images/map_western_cape_kruger.jpg Any idea? Thank you

Any update to solve this? Facing the same issue here

Okay the workaround I found is to convert as base64 first, thankfully worked like charm

ryanwolhuter commented 2 weeks ago

stuck here too, must be an issue with vercel. their guide literally describes doing exactly what we are doing, but it breaks with the error above https://vercel.com/guides/loading-static-file-nextjs-api-route

process.cwd() is returning /var/task/ when run on the vercel sever.

kamaladdi34 commented 1 week ago

Hey guys I also had this problem, try using an api route to read your file, It is the only way that worked for me. Happy Coding! 😊