Isolated FS trees, `chroot`

Hi, I am working on easywasi, which I recommend using with zenfs. It's been going good, and works well to give WASI wasm access to a filesystem, which allows for very light wasm, built without emscripten.

I would like to configure separate fs instances for each wasm loaded, so they can't interact with each other (like if you have multiple wasms in 1 page) but I am not sure how to do that.

Basically, I have a host+user-code (I call them "carts") that share an fs, but I want each instance of cart to not have access to other cart's fs.

Here is an example:

import { configure, fs } from '@zenfs/core'
import { Zip } from '@zenfs/zip'

await configure({
  mounts: {
    '/zip': { backend: Zip, data: await fetch('zip1.zip').then((r) => r.arrayBuffer())
    }
  }
})

// now fs has /zip from zip1.zip

// how to configure an fs2 with different mount at /zip ?
// this is other fs:
await configure({
  mounts: {
    '/zip': { backend: Zip, data: await fetch('zip2.zip').then((r) => r.arrayBuffer())
    }
  }
})

The main way I have thought about it so far is just to mount each cart at it's own dir, and not grab things in other dirs, but it would be better if each had their own.

@konsumer,

Thanks for opening an issue.

Much like node:fs and Linux, ZenFS works with a single file system tree.

Should you wish to have separate file system trees, you can import @zenfs/core multiple times. This will result in completely isolated trees. If you bundle your code, this will have 2 bundled instances of @zenfs/core.

One possible way you implement this:

import * as zenfs from '@zenfs/core';
import * as zenfsAlso '@zenfs/core?example=1';
import { Zip } from '@zenfs/zip';

await zenfs.configure({ ... });

await zenfsAlso.configure({ ... });

However, this could lead to additional issues. For example, @zenfs/zip imports from @zenfs/core, so some instanceof checks might fail. I've only verified that this is imported correctly, so use it at your own risk. I would like to support this correctly though, so if you try this and get errors due to different bindings, feel free to open an issue and I can improve it.

Please let me know if this resolves your issue.

Are you sure imports work like this? On node, I know multiple-imports are a single instance. I tried a test on web:

import {configure as m1} from 'https://esm.sh/@zenfs/core'
import {configure as m2} from 'https://esm.sh/@zenfs/core'

m1.tester = "if you see me on m2, it's shared"
console.log(m2.tester)

and it did log (since they are links to same instance.)

Update: ah, I see, you added a get-param. well, that is a solution that may work in some places. Seems less-than-ideal to download it twice, but that might work ok. At least in chrome, using esm.sh CDN, it did not work, and still logs, probably because they (like many CDNs like this) use http-redirects:

  import { configure as m1} from 'https://esm.sh/@zenfs/core?instance=1'
  import { configure as m2 } from 'https://esm.sh/@zenfs/core?instance=2'
  m1.tester = "if you see me on m2, it's shared"
  console.log(m2.tester)

1 idea might be to export a "default filesystem" (for backwards compatibility) but give user access to different instances. Maybe configure defaults to using a global fs, but it has an fs option, or is added to fs-object, and a way to create a new fs?

import { ZenFS } from 'https://esm.sh/@zenfs/core'

const fs1 = new ZenFS()
fs1.configure({...})

const fs2 = new ZenFS()
fs2.configure({...})

or maybe better:

const fs1 = new ZenFS({...})
const fs2 = new ZenFS({...})

It would take a lot of rework. I will think more about it.

Much like node:fs and Linux, ZenFS works with a single file system tree.

I hear you, but in node, you can still chroot, outside of runtime.

That brings me to another idea:

Maybe it could just be an out-of-lib helper that chroots:

import zenchroot from 'zenchroot'
import { configure, fs } from '@zenfs/core'
import { Zip } from '@zenfs/zip'

await configure({
  mounts: {
    '/zip1': { backend: Zip, data: await fetch('zip1.zip').then((r) => r.arrayBuffer())
    }
  }
})

await configure({
  mounts: {
    '/zip2': { backend: Zip, data: await fetch('zip2.zip').then((r) => r.arrayBuffer())
    }
  }
})

const fs1 = zenchroot(fs, '/zip1') 
const fs2 = zenchroot(fs, '/zip2')

The returned fs object would look just like zen's, but do path-resolution first (resolve full-path, don't allow outside of chroot, when you ask for /file you get /zip1/file.) I think I could set this up, if it's not something that seems like it should be in zen-fs/core.

Sidenote: I started work on some devices for zen. I currently have a working framebuffer (here is demo, and here is source.) Demo is 60-120fps, on web, on my tests, even though it's jerky, which is due to pauses in code. It works really well for wasi-sdk, and has performance comparable (maybe a bit better) to emscripten with SDL, which uses a ton more code (on both sides of wasm-barrier.)

Here is a static demo (click for white-noise) but I'm not sure I got the byte-order right, for sound.

I'd like to implement an oss-like /dev/dsp and /dev/input/event? for keyboard/mouse/gamepad.

With these basic drivers, it will be possible to make games using WASI, without having to add anything to wasm-side code, and very little to host, which is awesome!

1 idea might be to export a "default filesystem" (for backwards compatibility) but give user access to different instances. Maybe configure defaults to using a global fs, but it has an fs option, or is added to fs-object, and a way to create a new fs?

... new ZenFS() ...

ZenFS moved away from this approach a long time ago due to complexity and compatibility with Node.js. For example, we can't depend on any internal/instance state since users can import { readFileSync } from 'fs', drop in @zenfs/core as an alias for fs, and have it just work.

Since we can't depend on internal/instance state, there is not really a point to having a class. Additionally, it is a lot easier to implement Node.js functions as functions, rather than class methods.

With that out of the way...

I hear you, but in node, you can still chroot, outside of runtime.

That brings me to another idea:

Maybe it could just be an out-of-lib helper that chroots

chroot was my immediate go-to solution for this when I thought about it. I'll look into implementing this, since it looks like this will be both the simplest and most robust approach.

Sidenote: I started work ...

This is amazing! I'm glad that device file support has proven to be a useful feature. An output (rendering/screen/tty) device was one use case I thought of when working on devices, it's cool to see it implemented.

I'd like to implement an oss-like /dev/dsp and /dev/input/event? for keyboard/mouse/gamepad.

That would be great, no pressure though. While I was implementing devices, I thought it might be nice to have package (e.g. @zenfs/devices) for all of the more complex devices, that way they are accessible to more users. Let me know if you're interested in this. You can also reach out via email (on my Github profile) or Discord (@alsovo) if you want to chat privately.

With these basic drivers, it will be possible to make games using WASI ...

I originally got into programming because of my interest in game development, so I've come full circle =). Games on ZenFS would be stunning. I was looking into your work more, and came across this statement:

/mnt is a bit special in zenfs, and not traversed by a file-list, so if you want that, put it somewhere else

/mnt shouldn't be treated any differently. If you're experiencing different behavior that might be a bug. Please elaborate on this.

Since we can't depend on internal/instance state, there is not really a point to having a class. Additionally, it is a lot easier to implement Node.js functions as functions, rather than class methods.

Yep, totally agreed.

chroot was my immediate go-to solution for this when I thought about it. I'll look into implementing this, since it looks like this will be both the simplest and most robust approach.

Totally agree. Part of what I like about zen is it's simplicity and ease of modification. I can look into it, too. I think we really just need a super-solid "real path" implementation, so people can't escape the sandbox with things like /../../badstuff, and a "prefix" dir-router that won't let people get files out of that dir.

Let me know if you're interested in this.

Yes! That would be great. I need to think a bit more about the design. It also has some built-in limitations that are related to my usecase, like for example, I don't really want users to resize the screen (my game-engine is sort of like a "fantasy console", where you get what you get, but can make your game in any language.) It's partly that, but also I imagine a huge game-store/library, where you can browse games people made, and they all are the same lil size (similar to other fantasy consoles) and it would break down the feel if there was some that were a totally different resolution.

On linux, there is a whole complex ioctl system for saying "I want this resolution" but I will probly never implement that, since you can achieve the same thing by passing it a canvas of the right size, but the wasm-code itself has no control of that. It's much simpler without, but also I don't really want users to be able to control that stuff. Similar with OSS-like audio, like I don't really want users to change the sound-resolution on every game, it's just what it is (currently building to 48000 from 32bit floats, since that is web default, but that may change.)

Maybe keeping all the devices separate would be nice too, like maybe you only need graphics/sound/input, but not all 3. They are currently tied together, just because that is what I started with and it was faster to write (a single read/write for all.) I was being lazy, though, and they could definitely be pulled apart. I was really impressed with the performance of the framebuffer, and although I don't think I have the sound right yet, it makes whitenoise really fast!

Another idea here is removing the devices option and just having it as a totally separate "driver" (similar to zip) this would allow chroot tricks (like /cart1/dev/fb0) and allow people to put things wherever they want. Maybe all these devices could be combined into 1 package, that users can grab ala carte.

I think another big area here would also be testing on native framebuffer. I am basically guessing on the format, based on blog posts, but I would like it to roughly work like a real one, to make it easier to port native code (albeit old & outdated framebuffer code) to wasi.

I also need to learn more about your buffered setup & streaming. I think for audio, on oss, you can fill the buffer greedily, and it will play at a slower rate. Like you can run cat music.raw > /dev/dsp and it finishes quickly, but then takes as long as the file is to play.

I originally got into programming because of my interest in game development, so I've come full circle =). Games on ZenFS would be stunning.

me too! My goal for null0 is a light engine for trying out 2D game ideas quickly, in whatever language you like, and sharing it easily, without anyone having to recompile, or need any tools to play it. I have a native & web runtime for it, so it's easy to share a link to a game, but it will also run on very low-end devices (I target cheap Ambernic gameboy-type devices, and it runs well on pizero2w, but I have eventual goals for esp32, for example, for a < 5$ handheld that can run them!)

/mnt shouldn't be treated any differently. If you're experiencing different behavior that might be a bug. Please elaborate on this.

Ah, ok. For some reason, if I mounted things in /mnt they would not show up in a recursive list of root, when other things did. When I put it in /zip it worked, so I figured it must be special (like how you can mark filesystems non-traversable in linux.) Now that I think about it, it could just be that it's 2-levels deep, instead of 1 (like /zip is 1 dir away from root, and /mnt/cart is 2.) Maybe a more correct explanation is that {recursive: true} doesn't do what I think it will, like on node? Maybe this is a feature/bug I could add/fix.

Yeah, security is a top priority for the chroot. I'm considering a potential "bind" mount method, just like with the mount GNU/Linux command. This could allow you to still use devices. In the mean time, you can create another instance of DeviceFS.

I did a lot of reading into how Linux handles devices. I think it could be worthwhile to implement something like ioctl.

As for the /mnt issue, @mcandeia recently implemented recursive readdir support. He may have something to add.

Yeah, security is a top priority for the chroot.

Definitely.

I'm considering a potential "bind" mount method, just like with the mount GNU/Linux command.

I love that idea, and it would pair nicely with chroot, but would have other usecases, like efficiently mirroring the same subtree in 2 places.

As for the /mnt issue, @mcandeia recently implemented recursive readdir support. He may have something to add.

Ah, yeh, that makes sense. I made a minimal tester (demo).

I can make another issue. I was originally operating from the assumption of "it works as designed" so I did not report, but it sounds like it might be a bug.

In testing, it seems like I get a few issues, some of which do not seem related to recursive.

import { configure, fs as fsOrig } from '@zenfs/core'
import { Zip } from '@zenfs/zip'

const fs = fsOrig.promises

const data = await fetch('example-fake-zip-file-1mb.zip').then((r) => r.arrayBuffer())

await configure({
  mounts: {
    '/zip': { backend: Zip, data },
    '/mnt/zip': { backend: Zip, data },
    '/very/deep/structure/zip': { backend: Zip, data }
  }
})

console.log('non-recursive list of /zip')
console.log(await fs.readdir('/zip'))
// works ok

console.log('non-recursive list of /mnt/zip')
console.log(await fs.readdir('/mnt/zip'))
// works ok

console.log('non-recursive list of /very/deep/structure/zip')
console.log(await fs.readdir('/very/deep/structure/zip'))
// works ok

console.log('non-recursive list of /')
console.log(await fs.readdir('/'))
// incorrectly outputs only ['zip'] should be ['zip', 'mnt', 'very']

console.log('recursive list of /')
console.log(await fs.readdir('/', { recursive: true }))
// throws error about /zip missing

Normally, this is the sort of thing I would catch with tests, but the readdir test does not seem to be catching the issue. Maybe it's something with zip/web, or just that it's mounted and not directly written (as in tests) ?

Update: It appears to be due to not making the dirs that are to be mounted, but there is some incosisty with that. More discussion at #127

I will move discussion about devices over to web-zen-dev, which I am happy to transfer ownership (to zen-fs org) and add you as a maintainer.

zen-fs / core

Isolated FS trees, `chroot` #124