search

tinygo-org / tinygo

Go compiler for small places. Microcontrollers, WebAssembly (WASM/WASI), and command-line tools. Based on LLVM.
https://tinygo.org
Other
15.17k stars 889 forks source link

Wasm: Improve support of embedded systems with limited RAM #1694

Open vshymanskyy opened 3 years ago

vshymanskyy commented 3 years ago

The motivation about this request is the same, as described here: https://github.com/AssemblyScript/assemblyscript/issues/1089

Previously, we were using -heap-size 4096 to limit the (physically) required amount of linear memory. -heap-size was removed in #1567 , and now the default heap allocator is using the whole page (64KB) of memory.

P.S. Yes, TinyGo compiled to wasm can run on systems with as little as 16 KB of RAM :sunglasses: . Example: https://github.com/wasm3/wasm3-arduino/tree/master/wasm_apps/tinygo

aykevl commented 3 years ago

Previously, we were using -heap-size 4096 to limit the (physically) required amount of linear memory. -heap-size was removed in #1567 , and now the default heap allocator is using the whole page (64KB) of memory.

Actually, -heap-size=4096 would have allocated 64kB of RAM because that's the page size of WebAssembly, so nothing has changed here. The only difference is that it can now grow beyond that instead of being limited to 64kB. Maybe it just appeared to work before because the application didn't allocate a lot of memory?

vshymanskyy commented 3 years ago

@aykevl sorry I got sick ;( I'll need to dig into this further, but I'm pretty sure on 0.12.0 there was a difference between -heap-size=4096 and, say -heap-size=61440. Quick question: does the allocator try to put any internal markers or access memory on high addresses, maybe during the initialization? I'll try to provide a backtrace with 0.17.0 ASAP

aykevl commented 3 years ago

I'll need to dig into this further, but I'm pretty sure on 0.12.0 there was a difference between -heap-size=4096 and, say -heap-size=61440.

I'm pretty sure it did not, because it would round the number up to 64K bytes. A non-rounded number would not have been accepted by the WebAssembly linker.

Quick question: does the allocator try to put any internal markers or access memory on high addresses, maybe during the initialization?

Ah, I see. Yes, the first commit of https://github.com/tinygo-org/tinygo/pull/1567 moved the heap metadata from the start of the heap to the end of the heap. So I'm guessing it worked by accident before that PR because not much memory was allocated, but that PR broke it by moving the metadata to higher addresses.

As a workaround, you could use -gc=leaking which does not attempt to free any memory and also doesn't track which block of memory are allocated (apart from the "start of free area" pointer).

I'm not sure how this can be best solved, it's a rather specific use case after all. Note that you can also use TinyGo directly on many of the chips supported by wasm3, and I think that would be the preferred solution in most cases.

vshymanskyy commented 3 years ago

Yes, I know we can run native code as well, but there are many different use cases. In this case we have virtualuzation/SFI. I'm sure the dummy heap allocator will work, and covers the most simple cases. My question is, is there any way/will to make the heap size more granular (or just limit it to a specific size). Minimal alignment/step of 1K would be great.

On Wed, 10 Mar 2021, 15:55 Ayke, notifications@github.com wrote:

I'll need to dig into this further, but I'm pretty sure on 0.12.0 there was a difference between -heap-size=4096 and, say -heap-size=61440.

I'm pretty sure it did not, because it would round the number up to 64K bytes. A non-rounded number would not have been accepted by the WebAssembly linker.

Quick question: does the allocator try to put any internal markers or access memory on high addresses, maybe during the initialization?

Ah, I see. Yes, the first commit of #1567 https://github.com/tinygo-org/tinygo/pull/1567 moved the heap metadata from the start of the heap to the end of the heap. So I'm guessing it worked by accident before that PR because not much memory was allocated, but that PR broke it by moving the metadata to higher addresses.

As a workaround, you could use -gc=leaking which does not attempt to free any memory and also doesn't track which block of memory are allocated (apart from the "start of free area" pointer).

I'm not sure how this can be best solved, it's a rather specific use case after all. Note that you can also use TinyGo directly on many of the chips supported by wasm3, and I think that would be the preferred solution in most cases.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/tinygo-org/tinygo/issues/1694#issuecomment-795444747, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALP3FHPMG76DICHXRTNAJTTC524FANCNFSM4YXCKDYQ .