manuel-woelker / rust-vfs

A virtual filesystem for Rust
Apache License 2.0
372 stars 45 forks source link

`MemoryFS::new()` panics in webassembly #68

Open voltrevo opened 4 months ago

voltrevo commented 4 months ago

Hi. I was hoping to use vfs as an abstraction so that a lib designed for use with std::fs (that really just uses the filesystem for simple input) could be used from webassembly. I went through and adjusted everything to go through vfs instead, but then in the webassembly build I got a panic on MemoryFS::new() 😭.

Any idea why this would be and if it could be fixed?

manuel-woelker commented 4 months ago

Hi, interesting use case 👍

Do you have a stack trace for the panic? That might help diagnose the issue.

As of yet, webassembly is not supported as a target. It might be difficult to do, due to the constraints of the platform. But if it can be done with reasonable effort, I think it is worth considering.

I just gave it a quick try with

cargo test --target wasm32-wasi

Running the resulting binary with

wasmtime target/wasm32-wasi/debug/deps/vfs-4f7698969643a134.wasm  memory

Seems to run ok

test result: ok. 64 passed; 0 failed; 0 ignored; 0 measured; 332 filtered out

Running the whole test suite panics when it tries to get the temp directory in the physical fs tests. It looks like that is not supported in wasm.

manuel-woelker commented 4 months ago

Here's the experimental branch with some required version updates to allow compilation: https://github.com/manuel-woelker/rust-vfs/tree/mw/wasm-support

piotr-roslaniec commented 1 week ago

Hi @manuel-woelker. I came across a similar issue when using wasm32-unknown-unknown compilation target. I reproduced it here: https://github.com/piotr-roslaniec/rust-vfs-wasm32-unknown-unknown.

I believe it's caused by a dependency on std::time::Instant. Perhaps this is something that can be fixed with https://github.com/manuel-woelker/rust-vfs/issues/38. In my humble experience #![no_std] is often a pre-requisite for WASM-compatible crates, as corroborated by this guide.

manuel-woelker commented 1 week ago

Hi @piotr-roslaniec , thanks for the test case!

I am wondering if there are "platform differences" when running via wasmtime compared to wasm-pack. My last test with wasmtime seemed to be fine with the Instant call.

The issue with the InMemoryFs seems to be caused in SystemTime::now() here.

Since this timestamp is kinda bogus anyway, a workaround might be to set it to a constant timestamp instead. This might even improve determinism.

I am still not sure on the best course of action regarding #![no_std] support. It might be worth the effort to create "bespoke" Read & Write traits for the FS impls to be std agnostic, and then have implement the std corresponding traits conditionally (and maybe even the embedded-io traits).

manuel-woelker commented 1 week ago

Tried to reproduce locally, unfortunately the test fails to run. I think it is due to https://github.com/rustwasm/wasm-pack/issues/1241 :-(

I'll try to find a workaround later.

piotr-roslaniec commented 1 week ago

Hi @manuel-woelker, thank you for looking into this. I've added a Node.js test, it should make things easier to reproduce: https://github.com/piotr-roslaniec/rust-vfs-wasm32-unknown-unknown/commit/598c98af9bc89b2ef3eefdcc24f7485ac1bb12a4#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R14

I also found this drop-in replacement for <SystemTime|Instant>::now(). I haven't tested it yet, but I was planning on speed-running WASM compatibility next week.