Remote Hosting + Server side rendering

nhtyy commented 1 year ago

Im having trouble geting my app deployed, using rehype-pretty-code as a unified plugin w/ server side rending and next js

tested on netifly and vercel, bpth providers seem not to bundle it with node modules at server side...

I tried setting the CDN and moving the to public/ with no luck in production.

any thoughts?

nhtyy commented 1 year ago

i feel like this should work but does not

nhtyy commented 1 year ago

vercel shows its still trying to reach into node modules

nhtyy commented 1 year ago

https://github.com/vercel/next.js/issues/52711

people asking some question on vercel repo

nhtyy commented 1 year ago

lmfao not even going to speculate WHY this works, and server side version doesnt but

TLDR

vercel / netfily dont include shiki in node modules on deploy
using setCDN in the client side works but is super slow obviously
using setCDN in server side Doesnt work
static will always work cause next is built with all deps on vercel
need a way to make server side instance look in public

joinemm commented 1 year ago

I had the same issue trying to get ISR working and managed to fix it by adding this into my next.config.js

experimental: {
  outputFileTracingIncludes: {
    '/blog/[slug]': ['node_modules/shiki/**/*'],
  },
},

requires the newest version of next (13.4). This makes the shiki folder available for the route.

nhtyy commented 1 year ago

Works!!! tysm !!! @joinemm

alivault commented 1 year ago

The answer provided by joinemm above did not work for me.

I found this answer solved the whole mystery for me and made it all work: https://github.com/shikijs/shiki/issues/138#issuecomment-1057471160

basically in nextjs there are 3 states:

build-time, on server -> this is getStaticProps
run-time, on server -> this is getServerSideProps, or getStaticProps in case of ISR, which is this issue
run-time, on client

We want Shiki on getServerSideProps. This means:

ALL languages are available
ALL themes are available
NONE of them are bundled into client side

The trick is:

Copy shiki/themes and shiki/languages to somewhere outside of node_modules, maybe under lib/shiki
Touch these folders in a server side function (e.g. fs.readdirSync)
Done!

The key here is to let Vercel nft to know about the existence of Shiki/themes and shiki/languages so they are included in the production run-time

Sample code: https://github.com/thien-do/memos.pub/blob/a3babb1f149f05c43012278331f885d81f5fcfac/lib/mdx/plugins/code.ts

credit to thien-do

dm1nh commented 1 year ago

Should we call touchShikiPath() at page.js? I have a server component that renders mdx only runs if I tap a button. Next.js know about this at build time? I follow this but no luck.

msquinn commented 1 year ago

The answer provided by joinemm above did not work for me.

I found this answer solved the whole mystery for me and made it all work: shikijs/shiki#138 (comment)

basically in nextjs there are 3 states:

build-time, on server -> this is getStaticProps

run-time, on server -> this is getServerSideProps, or getStaticProps in case of ISR, which is this issue

run-time, on client

We want Shiki on getServerSideProps. This means:

ALL languages are available

ALL themes are available

NONE of them are bundled into client side

The trick is:

Copy shiki/themes and shiki/languages to somewhere outside of node_modules, maybe under lib/shiki

Touch these folders in a server side function (e.g. fs.readdirSync)

Done!

The key here is to let Vercel nft to know about the existence of Shiki/themes and shiki/languages so they are included in the production run-time

Sample code: https://github.com/thien-do/memos.pub/blob/a3babb1f149f05c43012278331f885d81f5fcfac/lib/mdx/plugins/code.ts

credit to thien-do

Been tinkering with this for a bit. My setup is trying to parse mdx using next-remote-mdx in a page in the app directory and I still can't get this to work for me. My code structure for the lib portion is exactly the same, and then I have a function called FetchMDXFiles that I call within my page component which contains:

// inside my fetchMDXFile();
  const { content } = await compileMDX({
            components: defaultComponents,
            source: mdx.data,
            options: {
              mdxOptions: {
                remarkPlugins: [remarkGfm],
                rehypePlugins: [
                  serializeClipboardContent,
                  ...(getMDXCode as any), // dealing with some ts-bs atm
                  deserializeClipboardContent,
                ],
              },
            },
          });

Then for my page I have:

async function MyPage() {
const MDX = await fetchMDXFile();
// ... more code

I would think that calling these functions within my app dir page would be sufficient to 'mark' the files, but nonetheless when it hits this line of code const getShikiPath = (): string => pathJoin(process.cwd(), 'lib/mdx/shiki'); from the example in the solution, it can't find the file. My folder structure for lib is exactly the same as in the example. If anyone has any thoughts or has gotten this working in app dir page with next-mdx-remote lmk, it's be greatly appreciated.

rehype-pretty / rehype-pretty-code

Remote Hosting + Server side rendering #98

TLDR