proposal: cmd/link: add wasminitmem and wasmmaxmem params

[gggrafff]

Proposal Details

Hello!

The problem

I am compiling my application to GOOS=wasip1 GOARCH=wasm and I want to control the wasm memory more accurate. wasm-ld provides the following parameters for this:

--initial-memory=<value>    Initial size of the linear memory
--max-memory=<value>        Maximum size of the linear memory

But go tool link doesn't allow me to set these values. And I cannot use the wasm-ld because -linkmode=external requires CGO but CGO doesn't support wasm.

Proposal

I propose to add the initial-memory and max-memory cli-parameters to go tool link

Solving

I think that a solving is not too hard and I wish I could help. I expect that the edit should be to pass a couple of constants from the cli-parameters to the module that is responsible for initial memory and max memory. But I looked into the linker code and didn't find where these values are set. Maybe you can tell me how best to solve this task.

[cherrymui]

If we do this, it would probably be -wasminitmem and -wasmmaxmem. The flag does not apply to other platforms, so it would have wasm in the name to be clear. Also, it can also apply to js/wasm, not just limited to wasip1 (so wasm, not wasi).

One question is whether these settings are better to be per-build or per-program. If it is per-program, i.e. for the same program you'd expect the same setting in most cases, maybe it can be directives in the main package, like //go:wasminitmem NNN. Per-build settings (i.e. linker flags) are more flexible, but they need to be set every time it is built.

While Wasm does allow setting max memory at build time, would it make sense to be a run time setting? (The initial memory size would have to be a build time setting.)

Could you explain more about your use cases? So we can see what is the best option here. Thanks.

[cherrymui]

cc @golang/wasm

[gggrafff]

@cherrymui thank you

The flag does not apply to other platforms, so it would have wasm in the name to be clear

good point

are better to be per-build or per-program

I agree with you that per-build settings (i.e. linker flags) are more flexible, I would be prefer this

would it make sense to be a run time setting?

You are right, max memory can be set in runtime, so this setting in the linker is not so important, but initial memory is necessary

Could you explain more about your use cases?

My users write small functions for data processing, my service compiles it into wasi and executes it. These are often very small functions that only need 2MB of memory, but go link makes initial memory much larger by default, so I get memory overruns dozens of times. I need the functions to start with a minimum amount of memory and increase it only if necessary. I use wasmtime-go to execute these.

[johanbrandhorst]

Is there an opportunity to just adjust the initial memory down? I don't know why it's high by default if we can allocate more as we go.

[cherrymui]

Currently it is set to the size of all global data + 16 MB. From the comment it is enough for the runtime to initialize without growing memory. I suppose we can lower it, though I'm not sure if we can grow memory at very early stage of runtime initialization.

If we lower the default, I think we still need the flag, so users who want the old behavior can change it back.

If the initial memory is set to be smaller than the global data size, it probably won't work. So we probably require it to be bigger than data size.

[rsc]

If we need max maybe the flags should be wasmminmem and wasmmaxmem.

[rsc]

This proposal has been added to the active column of the proposals project and will now be reviewed at the weekly proposal review meetings. — rsc for the proposal review group

[rsc]

Retitled to be -wasminitmem and -wasmmaxmem. Any other comments?