Global variables / linear memory

[tshort]

This includes supporting Julia global variables and wasm globals. I think a string constant will need to be allocated in wasm's linear memory.

Some sort of crude memory allocator may be needed. Rust has wee_alloc for this.

[tshort]

On memory allocation, Emscripten has emmalloc. It has a C api. We could pre-compile that into a wasm file and link that into Julia code.

Rust has wee_alloc as noted above. It doesn't have a C api, so we can't use it directly from a pre-compiled wasm file. There is an issue to create a C api, though. We could also port it to Julia.

[kripken]

Both of those are good options for a small allocator. If you need performance, though, then dlmalloc.c is far faster than those on stuff like allocating and freeing many small objects. (Not sure if relevant for you here, though!)

[tshort]

Good point on dlmalloc, Alon. A good option might be to stick to the C API with malloc, free, etc. Then, we could pre-compile emmalloc and dlmalloc, and the user can decide what they want to plug in.

[tshort]

I managed to compile emmalloc to wasm. This file still needs the following defined: sbrk, memset, memcpy, and _assert_fail. I haven't figured out where Emscripten defines sbrk, yet. These could all probably be hand-written in WASM.

[kripken]

Emscripten's sbrk is here:

https://github.com/kripken/emscripten/blob/1bf286afe8e930c8087993cf85110a6e3db142ae/src/library.js#L438

It's somewhat complicated because it handles multithreading, memory growth, and error handling. A simple sbrk that might be good enough for most use cases might not need those, and should be just a few lines - bump the pointer, basically.

[sjorn3]

I managed to get a fairly neat array summing example compiling by dropping the output of @code_wasm into a module with allocate and deallocate (and a few others) pinched from a compiled AssemblyScript file, and then calling it from wasm-ffi (which does indeed seem to be a very neat package @tshort)

    (func $main/get_sum (type 2) (param i32) (result i32)
      (local i32) (local i32) (local i32)
      (i32.const 0)
      (set_local 3)
      (i32.const 1)
      (set_local 2)
      (block
        (loop
          (get_local 2)
          (get_local 0)
          (call $~lib/array/Array<u32>#get:length)
          (i32.const 1)
          (i32.add)
          (i32.eq)
          (i32.eqz)
          (i32.eqz)
          (br_if 1)
          (get_local 0)
          (get_local 2)
          (call $~lib/array/Array<u32>#__get)
          (set_local 1)
          (get_local 2)
          (i32.const 1)
          (i32.add)
          (set_local 2)
          (get_local 3)
          (get_local 1)
          (i32.add)
          (set_local 3)
          (br 0)))
      (get_local 3)
      (return))

With this passing the length in is unnecessary, so the transformation from code_typed is much simpler/natural. The only hiccup is that AssemblyScript has 0 based indexing, but that's a simple enough fix.

Ideally we'd probably want to write these functions in Julia at some point and get them compiling, but for the time being dropping our compiled functions into a base module with all of these things already present doesn't seem like a bad way to go, at least until structs and arrays are working.

[tshort]

Great! I assume the length must be stored in the first part of the array then?

[tshort]

A tidied array example would make an interesting blog post!

[sjorn3]

Yeah that seems to be exactly what it's doing.

[sjorn3]

I could definitely give that a go.

[sjorn3]

The problem I seem to be running into (and can't currently think of a solution to) is that I think wasm-ffi takes the array you pass to it, allocates some space for it in memory and copies it in. This means that attempts to alter the array from within webassembly aren't seen on the javascript side, though operations that don't alter the memory work as expected.

I'm not sure what the best way to handle this is, as being able to alter the memory from within wasm seems important, even though it might be possible to defer all of that to the js side. I need to look in to the problem some more and see if this is actually where it's going wrong.

[tshort]

@sjorn3, do you have example code?

[tshort]

There are examples in wasm-ffi of returning arrays, at least for Rust and AssemblyScript. Maybe the modified arrays can be returned with Julia?

[sjorn3]

Here's a gist of an example: https://gist.github.com/sjorn3/193c231ace6e1f26cbb827d31e542631 It needs to be run on Chrome 67, but I can change that if need be.

Is the example here somewhere: https://demille.github.io/wasm-ffi/docs/assemblyscript/? I can't seem to find it. And setting the return type to 'array' seems to break things.

[tshort]

Here's where the docs are for returning an array in AssemblyScript. There's something similar below that for Rust.

https://github.com/demille/wasm-ffi#user-content-assemblyscript-array

[sjorn3]

Not as clean as I might like but I have gotten something working. When I try to use BigInt64s it gets strange, so I'll have to stick with Int32 arrays for the time being. I'll try and work what I've learned into a full array example. Also I think it should be possible to compile (or transpile) a julia struct definition to one of the wasm-ffi ones, which would be fairly nice.

I haven't tried out a two dimensional array yet, but if it isn't supported directly I think an array of structs could be used to make that work.