memory.discard prototype & design

[bvisness]

We on the SpiderMonkey team have created a prototype of memory.discard from this proposal. We've also created a document exploring the design space and the results of our research.

https://docs.google.com/document/d/131BCKkMSdKCeU51XJ52mIy5O4i60CIGMopSh8aWNQG8/edit?usp=sharing

Our prototype is available behind a flag in Firefox Nightly - you can try it here. It does not yet support shared memories, as we’re still researching how to handle this on Windows.

I look forward to everyone's comments and suggestions! This instruction feels good to us so far.

[dtig]

Thanks @bvisness for filing this!

The V8 prototype currently has an API only function for memory.discard(numPages) with a similar API to memory.grow(numPages) which only decommits the number of pages from the end of committed memory. This is definitely less flexible than the approach outlined in the doc, so it's great to have a couple of approaches to experiment with. One of the motivations for this was to keep it in line with JS RABs/GSABs, where ResizableArrayBuffers are allowed to shrink, but SharedArrayBuffers are only allowed to grow.

Explicit cc for @dschuff because we had previously discussed having a couple of different toolchain flags that support both the approaches so this can be experimented with.

There's a few questions in this design space that might be interesting to discuss:

• What happens when the memory that's discarded is accessed again? In the case where the length remains constant I assume this is equivalent to reading zero-filled memory, and non-trapping?
• I have a strong preference for these instructions to be aligned to Wasm memory pages, simply because relaxing that is somewhat hairy for our bookkeeping. For the traps currently implemented, how are they surfaced to JS?

[titzer]

+1 to page-alignment for these operations. I don't think we want to introduce a new instruction that is redundant with memory.fill[1], rather, an instruction specifically for pages, which gives applications more explicit control.

[1] AFAICT, an engine could transparently drop pages for very large memory.fill(0) requests.

[eqrion]

There's a few questions in this design space that might be interesting to discuss:

• What happens when the memory that's discarded is accessed again? In the case where the length remains constant I assume this is equivalent to reading zero-filled memory, and non-trapping?

Yes, it's completely valid to implement this as filling the whole region with zero bytes. This proposed design does not change the length of the memory at all, or introduce regions that will trap when accessed.

Behind the scenes, we're using OS specific API's which 'essentially' replace the OS pages with lazy-allocated zero pages. This reduces memory pressure on the system until the pages are accessed again. On some platforms the pages are re-allocated on read, others on write. This is pretty platform specific, and the most details are on the linked docs.

• I have a strong preference for these instructions to be aligned to Wasm memory pages, simply because relaxing that is somewhat hairy for our bookkeeping.

I agree that keeping the base/length byte lengths aligned to wasm pages is simplest. The only argument I could see for relaxing this somehow is if wasm pages are too big that users rarely ever have that much address space unused and ready to free, especially with heap fragmentation. In that case, it'd be beneficial to have some lower granularity that is closer to the OS page size. But this has other issues and we shouldn't consider this unless we hear it would help from users.

For the traps currently implemented, how are they surfaced to JS?

If the base/length are not aligned to wasm page size, then we emit a WebAssembly.RuntimeError. We do this for both the instruction and JS-API entry points.

[bvisness]

We have now implemented memory.discard for shared memories in SpiderMonkey. The document has been updated with our findings and recommendations, and our demo now supports shared memories as well.

Windows makes this difficult, since none of the available virtual memory APIs directly allow us to replace working set pages with demand-zero pages in a single call. However, using memset(0) and VirtualUnlock() achieves the desired result.

At this point, we'll try out our memory.discard instruction in some real programs and see what kinds of improvements we get.