Indexing WASM memory directly from JS stopped working with update

[dukc]

I just updated Spasm to the latest (It was 0.1.13 before) and LDC to 1.18 (was probably 1.16). I didn't update my .js bindings.

The code does initialize, and calling any D function that returns only an int or a float works. But for some reason, if the D code returns a pointer, trying to derefer it directly like this: spasm.heapi32s[dBornePointer / 4] seems to always return 0. Dereferencing and setting new values still works by defining these:

export extern (C) byte derefer_byte(byte* arg){ return *arg;}
export extern (C) ubyte derefer_ubyte(ubyte* arg){ return *arg;}
export extern (C) short derefer_short(short* arg){ return *arg;}
export extern (C) ushort derefer_ushort(ushort* arg){ return *arg;}
export extern (C) int derefer_int(int* arg){ return *arg;}
export extern (C) uint derefer_uint(uint* arg){ return *arg;}
export extern (C) long derefer_long(long* arg){ return *arg;}
export extern (C) ulong derefer_ulong(ulong* arg){ return *arg;}
export extern (C) float derefer_float(float* arg){ return *arg;}
export extern (C) double derefer_double(double* arg){ return *arg;}
export extern (C) size_t derefer_size_t(size_t* arg){ return *arg;}

export extern (C) byte set_byte(byte* to, byte arg){ return *to = arg;}
export extern (C) ubyte set_ubyte(ubyte* to, ubyte arg){ return *to = arg;}
export extern (C) short set_short(short* to, short arg){ return *to = arg;}
export extern (C) ushort set_ushort(ushort* to, ushort arg){ return *to = arg;}
export extern (C) int set_int(int* to, int arg){ return *to = arg;}
export extern (C) uint set_uint(uint* to, uint arg){ return *to = arg;}
export extern (C) long set_long(long* to, long arg){ return *to = arg;}
export extern (C) ulong set_ulong(ulong* to, ulong arg){ return *to = arg;}
export extern (C) float set_float(float* to, float arg){ return *to = arg;}
export extern (C) double set_double(double* to, double arg){ return *to = arg;}
export extern (C) size_t set_size_t(size_t* to, size_t arg){ return *to = arg;}

...and using them instead.

Do you have any idea how this happened? I noticed that the newer .js files would pass a __heap_base argument to _start(), even when it does not take any arguments... is this related?

[skoppe]

Normally, with every update you simply do a dub run spasm:bootstrap-webpack and a dub run spasm:webidl -- --bindgen to update all the javascript files to the latest version. But I know have a custom setup, so here it goes:

I suspect you are having issues with the adjustments I made to the memory. You can have a look at the CHANGELOG.md, specifically the item about setupMemory() under 0.2.0.

I changed the config to the linker to export memory instead of importing it. This allows the wasm module to start with as little as 2kb of memory and grow as required.

But, Javascript is unaware of whenever wasm grows its memory, and when it does the heap pointers in js land get invalidated. That is the reason of the setupMemory addition. You need to call that every time before you murk in the heap (or every time the heap grows). Spasm is conservative and just calls it every time it enters a JS binding function from wasm.

It is a known issue and most libraries deal with it in this way, sadly.

[dukc]

I changed the config to the linker to export memory instead of importing it. This allows the wasm module to start with as little as 2kb of memory and grow as required.

Wow, sounds cool. That explains why I couldn't find new WebAssembly.Memory({initial:16*16, maximum:16*16}) in the newer .js files.

Finished for today, but can't wait to change my own .js to do the same. Thanks.

[skoppe]

__heap_base is currently passed from javascript to inform the webassembly module of where its heap starts. I recently found a way to resolve that internally. I'll first get master green again and then fix that so that you don't have to pass that.

In the meantime, you do need to call spasm.rt.memory.alloc_init and pass the __heap_base. Otherwise the allocator will stomp all over the stack. That is, if you use spasm's allocator at all.

[dukc]

Success. I just moved initialization of the WASM buffers to happen after initialization of my module. Since my custom malloc gives memory off a static buffer, no extra memory is ever needed from JS interpreter point of view, and the library now seems to work in LDC 1.18. I like the idea that I can now specify all my memory size needs in D.

There was one small change in Spasm needed to get 1.18 working. I'll create a PR.

[skoppe]

If you want you can get the __heap_base in wasm by declaring it like so:

  extern(C)
  {
    /* Symbols created by the compiler/linker.
     */
    extern __gshared
    {
      size_t __data_end;  // the end of the data section (starts at address 1024).
      size_t __heap_base; // the place you can start allocating from
    }
  }

Also check the spasm.intrinsics module where you can grow memory or requests its current size. If you ever need to.