syscall/js: increase performance of Call, Invoke, and New by not allowing new slices to escape onto the heap

[finnbear]

What version of Go are you using (go version)?

$ go version
go version go1.14.4 linux/amd64

Does this issue reproduce with the latest release?

Yes, in go1.15beta1

What did you do?

I used js.Call, js.New, and js.Invoke with multiple arguments, hundreds of times per frame, 60 frames per second in order to implement my WebGL game.

What did you expect to see?

The above functions are absolutely essential for getting work done in js/wasm, as they are analogous to a simple function call. They should be optimized as such. No excess go heap memory should be allocated, no extra go garbage collection. After implementing a hack to fix the issue I will go on to describe (see below), here is a CPU profile of memory allocation:

Importantly, memory allocation is 8% of the total time and does not include makeArgs allocating any slices that end up on the heap

What did you see instead?

2+ new slices allocated on the heap per call to one of the above three functions, as the first few lines of the makeArgs function... https://github.com/golang/go/blob/60f78765022a59725121d3b800268adffe78bde3/src/syscall/js/js.go#L361-L363

...possibly are assumed to escape to the heap via... https://github.com/golang/go/blob/60f78765022a59725121d3b800268adffe78bde3/src/syscall/js/js.go#L405-L406 ...or valueNew/valueInvoke, or if go's escape analysis optimization doesn't work on the caller's stack.

A CPU profile shows the significant penalty associated with allocating too many slices (mallocgc now accounts for over 20%), and that doesn't even include the extra garbage collector load that claims a few FPS.

I thought of adding //go:noescape before each of the above and potentially other functions implemented in javascript, but I didn't get any improvement right away. Only my hack to makeArgs worked consistently:

const maxArgs = 9 // A better hack/fix would grow the slices automatically
var argVals = make([]Value, 0, maxArgs)
var argRefs = make([]ref, 0, maxArgs)

func makeArgs(args []interface{}) ([]Value, []ref) {
    for i, _ := range argVals {
        argVals[i] = Value{}
    }
    for i, _ := range argRefs { // I don't think this is strictly necessary
        argRefs[i] = 0
    }

    argVals = argVals[:len(args)]
    argRefs = argRefs[:len(args)]

    for i, arg := range args {
        v := ValueOf(arg)
        argVals[i] = v
        argRefs[i] = v.ref
    }
    return argVals, argRefs
}

Another potential solution, building on the above hack, would be to make makeArgs take slices, to put the values/refs in, as arguments (instead of returning new slices). The caller could deal with efficiently handling the memory. Maybe this could help go's escape analysis, in conjunction with //go:noescape.

Side note: Ideally, a solution would address the heap allocation of an interface for certain arguments. Hundreds of thousands get allocated before the garbage collector runs, causing the garbage collector to take 12-20ms.

[finnbear]

I've confirmed this and so on.

Excellent!

Do you have a plan to send a change?

I didn't before, but now that you mention the possibility, I'll read this and submit a PR at some point in the next few days. (I will make an attempt to only include //go:noescape when it is consequential, unlike my original patches)

[hajimehoshi]

Sure, I'm happy to review your PR!

I will make an attempt to only include //go:noescape when it is consequential, unlike my original patches

That'll be a great first step, and very easy to review!

[hajimehoshi]

@finnbear Any updates? The release freeze has come (today?), we cannot make it for Go 1.18 unfortunately.

[finnbear]

Any updates?

Yep, PR submitted: https://github.com/golang/go/pull/49799

The release freeze has come (today?), we cannot make it for Go 1.18 unfortunately.

I was under the impression that the release freeze starts November 1, that the PR would have been too late even 5 days ago when I said I would do it, and therefore that there was no rush. If I had known today was a deadline, I would have prioritized this differently.

Apparently, it is possible to get a freeze exception, especially for low risk, high reward changes, but whether this would apply is subjective and I'll leave that up to maintainers to decide.

[gopherbot]

Change https://golang.org/cl/367045 mentions this issue: syscall/js: allocate makeArgs slices on stack

[hajimehoshi]

I was under the impression that the release freeze starts November 1, that the PR would have been too late even 5 days ago when I said I would do it, and therefore that there was no rush. If I had known today was a deadline, I would have prioritized this differently.

Oh sorry, I was misunderstanding. Thank you for the PR!

[abrander]

I'm not sure if this is the correct place to raise this issue, forgive me if it's not.

I can understand the reasoning for removing the Wrapper case from ValueOf. But why remove the interface itself? It's incredible useful for types embedding js.Value.

The Wrapper type makes it possible to do stuff like this:

package a

import (
    "syscall/js"
)

func DoStuffInJavascriptLand(obj js.Wrapper) {
    js.Global().Call("some_js_function", obj.JSValue())
}

package b

import (
    "syscall/js"
)

type Thingie struct {
    js.Value
}

type AnotherThingie struct {
    js.Value
}

func NewThingie() *Thingie {
    return &Thingie{Value: js.Global.New("something")}
}

func NewAnotherThingie() *Thingie {
    return &AnotherThingie{Value: js.Global.New("something")}
}

package main

import (
    "a"
    "b"
)

func main() {
    t1 := b.NewThingie()
    t2 := b.NewAnotherThingie()

    a.DoStuffInJavascriptLand(t1)
    a.DoStuffInJavascriptLand(t2)
}

I would imagine we will see a lot of type Wrapper interface {JSValue() js.Value} scattered across packages after the removal of js.Wrapper.

Did I miss something?

[finnbear]

But why remove the interface itself? It's incredible useful for types embedding js.Value.

The reason is not a technical limitation, because as you point out, as long as the case is removed from ValueOf, the performance issue is resolved.

I think it has more to do with how confusing it would have been to keep the Wrapper interface without allowing it to work in Call, New, and Invoke. At that point, it would be more of a suggestion of how to organize code, without making the code any simpler.

Also, code using Wrapper will now refuse to compile, alerting the programmer that they have to do some refactoring. If Wrapper was kept, the code would compile but panic at runtime, which seems undesirable.

I would imagine we will see a lot of type Wrapper interface {JSValue() js.Value} scattered across packages after the removal of js.Wrapper.

You might be right, and if this happens, you could write a proposal to re-instate Wrapper.

However, if you briefly assume that Wrapper never existed and was never part of the Call, New, or Invoke apis, would it make sense to add it to the standard library (without it working in Call, New, or Invoke)? I doubt it would make sense.

[abrander]

I think it has more to do with how confusing it would have been to keep the Wrapper interface without allowing it to work in Call, New, and Invoke. At that point, it would be more of a suggestion of how to organize code, without making the code any simpler.

Maybe. But just for the record: ValueOf accepting a Wrapper was never documented anywhere as far as I know. If the documentation were your only guide, you wouldn't even know that ValueOf accepted a Wrapper.

Also, code using Wrapper will now refuse to compile, alerting the programmer that they have to do some refactoring. If Wrapper was kept, the code would compile but panic at runtime, which seems undesirable.

Code implicitly using js.Wrapper will refuse to compile, so far we agree. But all the code that relied upon the (undocumented) feature of ValueOf accepting a js.Wrapper will experience a runtime panic. The (non-)presence of the Wrapper interface changes nothing about this.

I would imagine we will see a lot of type Wrapper interface {JSValue() js.Value} scattered across packages after the removal of js.Wrapper. You might be right, and if this happens, you could write a proposal to re-instate Wrapper.

I wish I knew how :-)

I now realize it's worse than I imagined. The JSValue() method disappeared too. We can't even fix this by adding a Wrapper in every package.

However, if you briefly assume that Wrapper never existed and was never part of the Call, New, or Invoke apis, would it make sense to add it to the standard library (without it working in Call, New, or Invoke)? I doubt it would make sense.

I'm uncertain. It fitted a purpose - recognizing types backed by a js.Value. Either by embedding a js.Value or by satisfying the interface in another way. How can packages in any meaningful way express that they accept a js.Value wrapped in anything without a Wrapper-like interface post 1.18? That's how I saw js.Wrapper used.

My suggestion as of now would be to write a small wrapper for syscall/js itself, adding JSValue() and Wrapper. But that won't help inter-project calling much.

TLDR: js.Wrapper gave the whole community an interface for "everything backed by a javascript value". We don't have that anymore.

[finnbear]

I wish I knew how :-)

The good news is that submitting a proposal is as simple as creating a GitHub issue with a title similar to proposal: syscall/js: Reinstate js.Wrapper interface. The issue description would include a brief justification for the change to add back js.Wrapper.

You make some good points, so there is a chance your proposal would be accepted :)

[garet90]

Any updates on this and #49799 ?

[finnbear]

Any updates on this and #49799 ?

AFAIK the PR happened to coincide with a release freeze.

I haven't touched Go since submitting it, and It looks like a fairly trivial merge conflict materialized (interface{} -> any). I might attempt to fix it, but I have deleted the Go beta toolchain I used for the PR to save disk space, so hopefully testing locally isn't required.

If anyone wants to help move the PR forward, I'd be happy to grant access to my fork.

@garet90

[garet90]

Sure, we're currently in a release freeze right now but I'd be happy to help push it forward after

[finnbear]

Sure, we're currently in a release freeze right now but I'd be happy to help push it forward after

Cool, I've invited you to be a collaborator on my go fork, which should probably allow you to edit the PR (https://github.com/finnbear/go/tree/makeargs-stackslice). Feel free to pursue other options like making your own fork/PR with the same changes if it is easier.

[mokiat]

I am hitting this same problem. I am happy to see that someone has reported it already, though sad to see that it lingers for nearly 3 years 😢

In my case, I am making a large number of WebGL calls and the JS heap jumps from 2MB to 4MB in the timespan of 200ms-400ms. It gets GCed and then all over again, resulting in a jagged heap chart. This, combined with #54444 , causes the whole game to stutter and the CPU to spike. In contrast, the native implementation of the same game has a steady heap profile.

I tried to follow the thread above and as far as I can tell the #49799 MR should help fix this. I am not sure I understood how the js.Wrapper fits into all of this, though.

Is there any chance that with #57968 this would get more traction soon? I would be happy to lend my help, though I fear my understandings in the low-level inner workings of the Go runtime might not be up to the task yet.

EDIT: After a bit more troubleshooting, it appears that while this issue is related, the majority of the allocations might actually be occuring inside the wasm_exec.js file (syscall/js.valueInvoke and syscall/js.copyBytesToJS->subarray).

[unitoftime]

Hi I am also running into this same issue and would love to see it fixed.

This is from an 'alloc_space' memory profile in WASM (go version go1.22.1 linux/amd64), I'm seeing ~39% of my allocations happening in js.makeArgs, specifically at the two slice allocations flagged in the original issue description:

Happy to try and help, if I can! Also would be happy if there was a workaround to reduce my allocations by calling different APIs, but I couldn't find anything.

[finnbear]

@mokiat @unitoftime AFAIK, someone just needs to revive this PR which unfortunately landed during a release freeze. I've since migrated to Rust and don't necessarily have the Go toolchains installed or the relevant details in mind (e.g. the meaning of //go:wasmimport gojs syscall/js.funcName). Consider making a new PR with similar changes! I recommend applying the diff to your installed Go toolchain and recompiling your application to test.

[unitoftime]

Cool. Thanks for the info and initial research! I'll take a more in depth look next chance I get.

[gopherbot]

Change https://go.dev/cl/576575 mentions this issue: syscall/js: allocate arg slices on stack for small numbers of args