search

kawamuray / wasmtime-java

Java or JVM-language binding for Wasmtime
Apache License 2.0
127 stars 29 forks source link

Memory growth #23

Closed nicholasjackson closed 3 years ago

nicholasjackson commented 3 years ago

Hi, I don't think this is an issue with wasmtime-java however I would appreciate your thoughts on the below so that I can feed some info back to the TInyGo team.

With a TinyGo compiled application that opens a file, encrypts it using AES128, and then writes the file back to disk I get the following error when trying to write the file.

Caused by:
    0: failed to invoke command default
    1: wasm trap: out of bounds memory access
       wasm backtrace:
           0:  0x434 - <unknown>!strlen
           1: 0x2f65 - <unknown>!__wasilibc_find_relpath
           2: 0x32c9 - <unknown>!open
           3: 0x86f4 - <unknown>!_os.unixFilesystem_.OpenFile
           4: 0x8b18 - <unknown>!os.OpenFile
           5: 0xd728 - <unknown>!io/ioutil.WriteFile
           6: 0x54a6 - <unknown>!encrypt
           7: 0x4fbc - <unknown>!_start

Digging into this I found that it is not actually a problem with the file write in Wasi but the TinyGo applications use of the Wasm memory. My uneducated guess is that TinyGo is either not allocating enough memory for the application or that the callbacks to grow the memory are not failing silently. If I manually allocate more pages of memory than the 2 TinyGo allocates the problem disappears.

Initial TinyGo allocation 128Kb

  (table (;0;) 1 1 funcref)
  (memory (;0;) 2)
  (global (;0;) (mut i32) (i32.const 65536))

Modified to 1Mb

  (table (;0;) 1 1 funcref)
  (memory (;0;) 16)
  (global (;0;) (mut i32) (i32.const 65536))

Does this look like a bug in TinyGo?

go.zip

kawamuray commented 3 years ago

Yeah, I think I saw a similar issue before, with rust compiler. In rust compiler the memory size is decided based on stack allocation size, so by adjusting stack size in compiler option we can allocate bigger memory in the generated wasm module: https://github.com/kawamuray/decaton-wasm-playground/blob/master/wasm-processor-fileio/.cargo/config#L5

I'm not sure if tinygo has similar build option tho. Alternatively, maybe it's possible to grow memory size after loading it through wasmtime API, so you don't have to modify wasm module itself: https://github.com/kawamuray/wasmtime-java/blob/master/src/main/java/io/github/kawamuray/wasmtime/Memory.java#L29

nicholasjackson commented 3 years ago

Thanks so much, @kawamuray I checked Memory.grow and it still returns the same error. Once I have this presentation out of the way I am going to raise these issues with the TinyGo team.

I have for now got a pretty sweet proof of concept running where I can create a WASM module in Go that uses go`s crypto libraries to encrypt and decrypt files, this is then consumed from a Rust WASM module. Strings are always fun but never more so when memory is not shared across modules. A couple of creative hacks and a host function later and that problem is solved.

I don't suppose you know of any resources that have any status on the situation with Berkley sockets and WASM?

Kind regards,

Nic

kawamuray commented 3 years ago

I have for now got a pretty sweet proof of concept running

Glad to hear that :)

I don't suppose you know of any resources that have any status on the situation with Berkley sockets and WASM?

I've been watching some issues like https://github.com/WebAssembly/WASI/pull/312 and https://github.com/bytecodealliance/wasmtime/issues/70 but I think nothing usable has shipped yet.