Unicode emojis are being escaped

mdx-bundler version: 10.0.2
node version: v18.17.0
npm version: 9.6.7
yarn version: 4.0.2

Relevant code or config

intro.md

---
title: intro
---

Smiley Face 😀

mdxcollection.ts

async function prepareFile (
  options: Partial<Parameters<typeof bundleMDX>[0]>,
  filePath: string
) {
    const fileContents = await fs.readFile(filePath, 'utf-8');
    log('fileContents: ', fileContents)

    const source = String(fileContents);
    log('source: ', source)

    const options = Object.assign(
      {},
      Object.create(options),
      {
        source,
      }
    );

    const contents = await bundleMDX(options);
    log('contents.code: ', contents.code)

    return content
}

What you did:

mdxcollection('intro.md')

What happened:

mdxcollection('intro.md')

// fileContents: '---\ntitle: intro\n---\n\nSmiley Face 😀\n'

// source: '---\ntitle: intro\n---\n\nSmiley Face 😀\n'

// contents.code: "var Component=(()=>{var p=Object.create;var r=Object.defineProperty;var f=Object.getOwnPropertyDescriptor;var l=Object.getOwnPropertyNames;var _=Object.getPrototypeOf,d=Object.prototype.hasOwnProperty;var j=(t,n)=>()=>(n||t((n={exports:{}}).exports,n),n.exports),M=(t,n)=>{for(var e in n)r(t,e,{get:n[e],enumerable:!0})},m=(t,n,e,i)=>{if(n&&typeof n==\"object\"||typeof n==\"function\")for(let o of l(n))!d.call(t,o)&&o!==e&&r(t,o,{get:()=>n[o],enumerable:!(i=f(n,o))||i.enumerable});return t};var h=(t,n,e)=>(e=t!=null?p(_(t)):{},m(n||!t||!t.__esModule?r(e,\"default\",{value:t,enumerable:!0}):e,t)),y=t=>m(r({},\"__esModule\",{value:!0}),t);var a=j((w,s)=>{s.exports=_jsx_runtime});var D={};M(D,{default:()=>x,frontmatter:()=>C});var c=h(a()),C={title:\"intro\"};function u(t){let n={p:\"p\",...t.components};return(0,c.jsx)(n.p,{children:\"Smiley Face \\u{1F600}\"})}function x(t={}){let{wrapper:n}=t.components||{};return n?(0,c.jsx)(n,{...t,children:(0,c.jsx)(u,{...t})}):u(t)}return y(D);})();\n;return Component;"

[!NOTE]

children:\"Smiley Face \\u{1F600}\"

So this doesn't end up rendering the emoji correctly anymore.

Reproduction repository:

I tried reproducing it here, but inexplicably it wont (perhaps because it's an in memory filesystem on stackblitz?).

https://stackblitz.com/edit/remix-run-remix-cqytcp?file=app%2Froutes%2F_index.tsx

Problem description:

unicode emoji is escaped.

Suggested solution:

🤷🏻

solution: don't forget <meta charSet="utf-8" /> in your head.

By default esbuild's output is ASCII-only. Any non-ASCII characters are escaped using backslash escape sequences. One reason is because non-ASCII characters are misinterpreted by the browser by default, which causes confusion. You have to explicitly add to your HTML or serve it with the correct Content-Type header for the browser to not mangle your code.

https://esbuild.github.io/api/#charset

so in my case: Remix, which means:

root.tsx

export function Layout({ children }: { children: React.ReactNode }) {
  return (
    <html lang="en">
      <head>
        <meta charSet="utf-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
        <Meta />
        <Links />
      </head>
      <body>
        {children}
        <ScrollRestoration />
        <Scripts />
      </body>
    </html>
  );
}

Remove <meta charSet="utf-8" /> and the problem appears.

Obviously it's not a problem with bundler-mdx, but I guess it was unexpected behaviour; should we document this in the README?

kentcdodds / mdx-bundler

Unicode emojis are being escaped #231