fix(core): simplify cloudflare wasm loading

[hi-ogawa]

Description

• Closes https://github.com/shikijs/shiki/issues/604

It looks like the exact usage of loadWasm(import('shiki/onig.wasm')) shown in the documentation doesn't work currently https://shiki.style/guide/install#cloudflare-workers.

There is actually a different way to load wasm directly WebAssembly.instantiate as demonstrated in packages/shiki/test/cf.ts and it works. However, this usage seems a little more convoluted, so I'm suggesting to improve loadWasm utility to support loadWasm(import('shiki/onig.wasm') usage like in the documentation.

https://github.com/shikijs/shiki/blob/044181dfd536f693dc0b7309db60837834182f89/packages/shiki/test/cf.ts#L11

I tested locally with pnpm -C packages/shiki test:cf, but It looks like this is not checked on CI. Please let me know if it's desired to have some integration test for cloudflare usage.

Linked Issues

https://github.com/shikijs/shiki/issues/604

Additional context

I was experimenting with shiki on Vite SSR demo https://github.com/hi-ogawa/reproductions/tree/main/shiki-604-cloudflare and currently my loadWasm looks like a following since I need to switch to import("shiki/onig.wasm") only on cloudflare build:

    await loadWasm(async (info) => {
        if (import.meta.env.VITE_BUILD_CF) {
            const mod = await import("shiki/onig.wasm" as string);
            return WebAssembly.instantiate(mod.default, info);
        } else {
            return import("shiki/wasm");
        }
    });

With this PR's patch, I would be able to always write await loadWasm(import("shiki/wasm")) then setup alias or custom resolution to switch to "shiki/onig.wasm https://github.com/hi-ogawa/reproductions/pull/7.

[netlify[bot]]

✅ Deploy Preview for shiki-matsu ready!

Name	Link
🔨 Latest commit	9ae6fc27cba826b5b8018f892376d1ef46e43450
🔍 Latest deploy log	https://app.netlify.com/sites/shiki-matsu/deploys/663c568e090374000877f2dc
😎 Deploy Preview	https://deploy-preview-670--shiki-matsu.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[netlify[bot]]

✅ Deploy Preview for shiki-next ready!

Name	Link
🔨 Latest commit	9ae6fc27cba826b5b8018f892376d1ef46e43450
🔍 Latest deploy log	https://app.netlify.com/sites/shiki-next/deploys/663c568edd6caa00080827ac
😎 Deploy Preview	https://deploy-preview-670--shiki-next.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[wouterds]

Tried all sorts of different ways but unable to get this working on Cloudflare with 1.5.2. Is there a Cloudflare example to plug & play deploy?

await loadWasm(import('shiki/wasm'));

const highlighter = await getHighlighterCore({
  themes: [import('shiki/themes/github-light.mjs')],
  langs: [
    import('shiki/langs/javascript.mjs'),
    import('shiki/langs/xml.mjs'),
    import('shiki/langs/java.mjs'),
    import('shiki/langs/bash.mjs'),
    import('shiki/langs/bibtex.mjs'),
  ],
});

results in

✘ [ERROR] service core:user:worker: Uncaught TypeError: Cannot read properties of undefined (reading 'buffer')

[hi-ogawa]

@wouterds Did you follow https://shiki.style/guide/install#cloudflare-workers? Note that import('shiki/wasm') is not intended to work but you need import('shiki/onig.wasm').

As written in the description, there is https://github.com/shikijs/shiki/blob/main/packages/shiki/test/cf.ts to test it out locally. Also I just tested it out in a fresh project https://github.com/hi-ogawa/reproductions/tree/main/shiki-670-cloudflare and it looks working for me.

[wouterds]

Sorry I should have clarified that I'm using Remix with Cloudflare Pages. Eventually I got it working in a similar fashion https://github.com/wouterds/wouterds.be/commit/fedec70bb3c510a26b2abdbba826ce80445d8d3e locally. However, my deploy to Cloudflare "succeeds" (wrangler pages deploy gives no error), but then the function fails with an unknown error (8000068) and the Wrangler logs don't tell a lot more.

✘ [ERROR] A request to the Cloudflare API (/accounts/8406f1e07e7516b76b5c2bf6f2b704d8/pages/projects/wouterds/deployments/05465991-1a75-4e90-8ae0-4e92d5743372/tails) failed.

  workers.api.error.unknown [code: 8000068]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

I would assume it's some size limitation on the free plan, but even that seems weird as the total size of the build folder is around 3 MB and the biggest file 300 KB, which is well below Cloudflare stated limits.

Edit: hopefully this gives some more insights once merged https://github.com/cloudflare/workers-sdk/pull/5819

Edit 2: turns out it indeed the bundle was too large

Edit 3: managed to work around it by loading shiki from esm.sh https://github.com/wouterds/wouterds.be/commit/aba12a2ec1a27c5f44f6c67416cc2f365ada4e8a

Edit 4: wrote about it https://wouterds.be/blog/code-highlighting-with-shiki-keeping-your-cloudflare-worker-bundle-small