Embed directory in executable with `bun build --compile`

[robobun]

The documentation mentions that I can embed a file in an executable produced by Bun (bun build --compile) by importing it, but is it possible to include a whole directory?

I want to build an Express app as an executable and use express.static() with browser assets. I'm looking for something like pkg's assets flag: https://github.com/vercel/pkg#assets

Originally reported on Discord: Embed folder in executable

[lukechilds]

@jaredpalmer are there any plans to support this? We have the exact same use case (express.static() from compiled binary).

[mayankchhabra]

+1 for this @jaredpalmer 🙏

[BLucky-gh]

I feelike that would be pretty annoying to implement, cause there's no "directory" class, so there's nothing you can necessarily import the directory as. And the alternative of it returning a path prefix that, when read with thr usual methods (with bun.file or node functions) would intercept the read call and return the bundled file's contents also feels too hacky, with maybe some sort of bundled_folder:// protocol paths being a less hacky option.

I think for the purposes of static file serving you might just need to get the full directory tree of the folder and import all of the files one by one (probably with dynamic import) at build time with a bun macro, and iterate over it at runtime (or to be more precise, server initialization time) to return a given file based on its path (basically implementing a worse version of express.static by hand)

[BLucky-gh]

Also to the 2 people above me: looks likr you're pinging the wrong Jar(r)ed, if you mean the creator of bun that would be @Jarred-Sumner, Jared Palmer is the VP of AI at vercel

[lukechilds]

you're pinging the wrong Jar(r)ed

Oops, thanks for pointing that out @BLucky-gh!

the alternative of it returning a path prefix that, when read with thr usual methods (with bun.file or node functions) would intercept the read call and return the bundled file's contents also feels too hacky, with maybe some sort of bundled_folder:// protocol paths being a less hacky option.

Out of interest why does intercepting fs calls feel hacky? I believe that's how pkg does it for Node.js and it works well. It often breaks between Node.js versions since they have to maintain external patches but it seems like that wouldn't be an issue here if it's directly built into Bun.

[BLucky-gh]

I checked out how pkg does it, and it requires that you specifically path.join(__dirname, '../path/to/asset') for the asset to be bundled and accessible at runtime, so if you access the file via any other way, it will not be intercepted. Also the way it does it is virtually mount the assets folder at /snapshot/, which is a fine workaround for a third-party package, but feels like quite the hack for a first-party feature within the runtime, esp considering that in addition to __dirname, there's import.meta.dir and in addition to node:fs there's also Bun.file, and other stuff to also keep in mind, intercept and maintain all of the interception code

[lukechilds]

Thanks for taking a look. Ah was not aware of the path quirks, yeah that is not ideal. I guess Bun could do everything in memory instead of writing out to disk but yeah there will still be interception code and path awkwardness to handle. Potentially node:fs and Bun.file can both be handled together with a lower level abstraction but I'm not familiar with Bun internals so maybe that's not possible.

There is also Leaf (https://deno.land/x/leaf@v1.0.4) for Deno but also feels quite hacky.

Also re the fs mounting, I agree it does sound kinda hacky but that's also how AppImage works and it seems to be pretty robust.

[BLucky-gh]

Also re the fs mounting, I agree it does sound kinda hacky but that's also how AppImage works and it seems to be pretty robust.

AppImages are mostly intended for packaging applications for end users, and thus have higher system requirements. Older systems with old versions of glibc for example can't run it because AppImages don't bundle their own libc, and they can't run on docker without a hack to either disable some of the sandbox protections or unpacking the appimage and running it from an actual folder, among other issues

[seveibar]

I'd like a first class solution to this, but I built a module called make-vfs to hack around the problem with a build step.

e.g.

bunx make-vfs --dir ./routes --content-format import-star --outfile static-routes.js

Creates something that can be statically imported:

import * as _api_dev_package_examples_create from "./../routes/api/dev_package_examples/create"
import * as _api_dev_package_examples_get from "./../routes/api/dev_package_examples/get"
import * as _api_health from "./../routes/api/health"

export default {
  "api/dev_package_examples/create": _api_dev_package_examples_create,
  "api/dev_package_examples/get": _api_dev_package_examples_get,
  "api/health": _api_health
}

A first class solution to this would be awesome though!

[xkkx]

I'd like a first class solution to this, but I built a module called make-vfs to hack around the problem with a build step.

e.g.

bunx make-vfs --dir ./routes --content-format import-star --outfile static-routes.js

Creates something that can be statically imported:

import * as _api_dev_package_examples_create from "./../routes/api/dev_package_examples/create"
import * as _api_dev_package_examples_get from "./../routes/api/dev_package_examples/get"
import * as _api_health from "./../routes/api/health"

export default {
  "api/dev_package_examples/create": _api_dev_package_examples_create,
  "api/dev_package_examples/get": _api_dev_package_examples_get,
  "api/health": _api_health
}

A first class solution to this would be awesome though!

And how to actually use your module? Would you be so kind to provide usage example either here, or maybe better in your module's repo? I managed to generate an output with your module but I don't see how I can use it

[seveibar]

@xkkx I've added details to the README, here's an example of using the generated static routes file with Bun:

// server.ts
import staticRoutes from "./static-routes"

Bun.serve({
  fetch(req) {
    const url = new URL(req.url);
    const path = url.pathname.slice(1); // Remove leading slash

    if (!(path in staticRoutes)) {
      return new Response("Not found", { status: 404 });
    }

    const route = staticRoutes[path]
    return route.default(req)
  },
  port: 3000,
})

Feel free to open issues on that repo as well- I don't want to take away from people potentially discussing a native solution! Thanks for trying it out 😀

Also note that if you don't have requirements around bundling you may be able to use Bun's builtin FileSystemRouter

[zjtakato]

+1

[clddup]

+1