esbuild-wasm: Import modules in the Browser | Documentation

[sinclairzx81]

Hi, I was wondering if it would be possible to include a little bit of documentation on how to use esbuild-wasm to bundle from source tree. This with goal of enabling source assets to be resolved and imported from arbitrary data stores (such as Http, LocalStorage or IndexedDB).

Given that the wasm version is synchronous, I was imagining something like the following where a source tree could be resolved up front (asynchronously by the host) prior to passing over to esbuild to bundle synchronously (with esbuild sourcing script assets from the tree data structure). Something like...

const tree = {
    'src/index.ts': (`
         import { Foo } from './foo'

         const foo = new Foo()
    `),
    'src/foo.ts': (`
        export class Foo { }
    `)
}

const bundle = esbuild.build('src/index.ts', { tree })

I wasn't certain if this functionality already exists, but couldn't locate information in the official documentation on how to handle this. From memory, I think TypeScript allows one to implement a IOHost interface that can be passed to the compiler, (allowing it to resolve assets independent of the host), so was imaging this is likely possible already with esbuild.

Many Thanks

[hyrious]

This can be made by writing a plugin, which returns absolute path in onResolve callback, and returns file contents in onLoad callback. Like such:

onResolve({ filter: /.*/ }, args => {
  // here i just replace the prefix './' with '/',
  // but normally you have to implement the resolver correctly
  return { path: '/' + args.path.replace(/^\.\//, '') }
})
onLoad({ filter: /.*/ }, args => {
  return { contents: tree[args.path], loader: 'default' }
})

There's an working example in my repo: https://github.com/hyrious/esbuild-repl/blob/main/src/stores/build.ts#L36-L59

[sinclairzx81]

@hyrious Hey, thanks for this! I do still wonder if it would be worth adding some official documentation around this approach. I think for browser implementations, resolving script assets for a bundle would be one of the first things people look for (as browsers can resolve such assets several dozen ways with none of those approaches actually fetching them from the file system).

I'll give this plugin approach a test run a bit later in the week, and report back if I run into any problems. Thanks again!

[sinclairzx81]

@hyrious Hiya, have tried implementing module resolution via plugins as per suggestion, but have run into some problems. The esbuild.transform(...) function works as expected (single file), however I'm having some issues getting esbuild.build(options) to work.

The following repro creates a plugin that filters for a default entry module (in this case index.js) via build.onLoad(...). The esbuild.build(...) call specifies index.js for its entryPoints with the expectation that the plugin would intercept index.js and resolve it's content.

import * as esbuild from 'esbuild-wasm'

// -----------------------------------------------------------
// Plugin
// -----------------------------------------------------------

function createPlugin(entry: string, contents: string): esbuild.Plugin {

    return {
        name: 'example',

        async setup(build: esbuild.PluginBuild) {

            // Filter for entry module and return contents. The namespace 'x' informs
            // esbuild that it should not attempt to resolve from the file system.
            build.onLoad({ filter: new RegExp(entry), namespace: 'x' }, async (args: any) => {

                console.log('onLoad', args)

                return { contents }
            })
        }
    }
}

// -----------------------------------------------------------
// Example
// -----------------------------------------------------------

await esbuild.initialize({ wasmURL: 'esbuild.wasm' })

const plugin = createPlugin('index.js', `console.log('hello world')`)

const result = await esbuild.build({
    entryPoints: ['index.js'], // Specify index.js as entry point. This should
    bundle: true,              // be intercepted by the plugins onLoad() callback.
    write: false,
    plugins: [plugin]
})

However I get the following browser errors with the onLoad() function never called.

✘ [ERROR] Cannot read directory ".": not implemented on js
✘ [ERROR] Cannot read directory ".": not implemented on js
✘ [ERROR] Could not resolve "index.js"

The implementation is somewhat derived from an example located https://codesandbox.io/s/h4xfy?file=/src/bundler/index.ts which does work, however I notice the initialization of esbuild-wasm is handled via .startService() and not initialize() as is suggested by the current documentation. The startService() function is not part of the definitions I have for "esbuild-wasm": "^0.14.13" where I notice their example uses "esbuild-wasm": "0.8.27". So not sure if there's been changes in recent revisions.

I feel like I might be missing something obvious, do you see anything that stands out as wrong in this implementation?

[hyrious]

1. you don't have to (and shouldn't) provide a namespace to onLoad unless you have resolved one with namespace, i.e. the expected code pattern is like this:

   onResolve({ filter }, args => {
   return { path: '/index.js', namespace: 'x' }
   })
   onLoad({ filter, namespace: 'x' }, args => {
   return { contents }
   })

However, although this approach allows you to resolve any non-absolute path in virtual namespaces, it actually changes the error message to under the namespace, which looks not good (https://github.com/hyrious/esbuild-repl/issues/7#issuecomment-1009497883).

2. onResolve is needed, otherwise it will perform the default behavior, i.e. searching files in real fs, which will crash in the browser.

3. startService() is an old API and was replaced by initialize(), just use it. Besides, you appeared to not providing a correct wasmURL, make sure this url can be fetch()-ed in the browser.

[sinclairzx81]

@hyrious Thanks again for your help. You were quite right about needing the onResolve and ensuring the paths were absolute. I have a fairly naive implementation working which does the virtual source tree as per original comment, as follows.

import * as esbuild from 'esbuild-wasm'
import { Path } from './path'

function customResolver(tree: Record<string, string>): esbuild.Plugin {

    const map = new Map(Object.entries(tree))

    return {

        name: 'example',

        setup: (build: esbuild.PluginBuild) => {

            build.onResolve({ filter: /.*/, }, (args: esbuild.OnResolveArgs) => {

                if (args.kind === 'entry-point') {

                    return { path: '/' + args.path }
                }

                if (args.kind === 'import-statement') {

                    const dirname = Path.dirname(args.importer)

                    const path = Path.join(dirname, args.path)

                    return { path }

                }

                throw Error('not resolvable')

            })

            build.onLoad({ filter: /.*/ }, (args: esbuild.OnLoadArgs) => {

                if (!map.has(args.path)) {

                    throw Error('not loadable')
                }
                const ext = Path.extname(args.path)

                const contents = map.get(args.path)!

                const loader = (ext === '.ts')  ? 'ts'  : 
                               (ext === '.tsx') ? 'tsx' :
                               (ext === '.js')  ? 'js'  :
                               (ext === '.jsx') ? 'jsx' :
                               'default'

                return { contents, loader }
            })
        }
    }
}

// -----------------------------------------------------------
// Example
// -----------------------------------------------------------

await esbuild.initialize({ wasmURL: 'esbuild.wasm' })

const tree = {

    '/util/encode.ts': `

        export function encode(data: string): Uint8Array {

            return new Uint8Array(1)
        } 
    `,
    '/lib/foo.ts': `

        import { encode } from '../util/encode.ts'

        export function foo() {

           return encode('foo')
        }
    `,
    '/lib/bar.ts': `

        import { encode } from '../util/encode.ts'

        export function bar() {

           return encode('bar')
        }
    `,
    '/lib/index.ts': `

        export * from './foo.ts'

        export * from './bar.ts'
    `,
    '/index.ts': `

        import { foo, bar } from './lib/index.ts'

        foo()

        bar()
    `
}

const result = await esbuild.build({
    entryPoints: ['index.ts'],
    plugins: [customResolver(tree)],
    bundle: true,
    write: false,
})

const decoder = new TextDecoder()

console.log(decoder.decode(result.outputFiles[0].contents));

// outputs: (() => {
//     // util/encode.ts
//     function encode(data) {
//         return new Uint8Array(1);
//     }

//     // lib/foo.ts
//     function foo() {
//         return encode("foo");
//     }

//     // lib/bar.ts
//     function bar() {
//         return encode("bar");
//     }

//     // index.ts
//     foo();
//     bar();
// })();

So this appears to be working as expected which is great. Still wondering if it's worthwhile including a little bit of documentation on the esbuild-wasm side as it's not immediately obvious that one should consider plugins for module resolution when working in a browser context (tho that might just be me as I've not spent much time building plugins for esbuild even in the typical case, plugins may just be par for the course for handling this kind of thing).

[sinclairzx81]

Might close off this issue. I'm guessing most experienced users are probably already quite familiar with using plugins for asset resolution (as this would be a typical usecase for plugins), so would imagine most users would look towards plugins in the esbuild-wasm scenario also (with plugin documentation already very good).

I still kind of think that maybe a mention to plugins being the primarily mechanism for module resolution in the wasm case would be helpful to the uninitiated, but guessing people would probably arrive there on their own after thoroughly reading through the plugin documentation.

Example code for handling the simple case is above should someone stumble across this thread. Thanks again for your help!

[1SAURABHKUMAR1]

Hi sinclairzx81 . Can you please share those code in sandbox . I am trying to build a online code editor

[sinclairzx81]

@1SAURABHKUMAR1 Hello. The code above is the entirety of the setup. You should be able to copy and paste it into a sandbox of your choosing. The Path import is a browser compatible version of Node's path core module. You should be able to source these functions from the web.

[1SAURABHKUMAR1]

@sinclairzx81 Thanks for information . I was a bit confused about Path module

[1SAURABHKUMAR1]

@sinclairzx81 Can you help me to convert this piece of code into import via file path and also import via skypack or any other cdn

[sinclairzx81]

@1SAURABHKUMAR1 See reference project here https://github.com/sinclairzx81/esbuild-wasm-resolve

You can either clone and customize it or just take it as a dependency. See example project included for usage.

Hope that helps S

[MarketingPip]

@sinclairzx81 - I thank you so much. I have been looking for this forever.