Screen glitching and long `roc_alloc` call times

[Walexander]

Hello - Having a great time writing my Real-Time-Strategy game in roc-wasm4! However, when my update function makes multiple roc_alloc calls, the ensuing frame "glitches", showing garbage on parts of the screen.

Below is a profile showing the issue. Most frames generated are "clean" like the two on the left. You can see the "glitch" in the right-most frame -- there's noise over the game's text at the bottom of the screen.

This always happens after update runs this A* search function (which, in turn, triggers a number of roc_alloc calls). Here is a zoomed-in version of the update call from the previous screenshot. As you can see, the recursion is fairly deep but not at all wide -- the vast majority of time is spent in the bit_set_ArrayBitSet.isSet call.

I've also found the glitching:

• Happens both natively and in the browser.
• Gets worse (covers more of the screen, shows more noise) the more roc_alloc calls update makes.
• Reflects what's actually in the frame buffer memory (calling getPixel() on a noisy pixels returns the noise color).

Should I simply get better at managing allocations? Or is this potentially a bug in roc-wasm4, roc, wasm4, zig or wasm?

[bhansconnect]

First question, is this built with optimization on? This may be a case where simply enabling optimization would fix the issue.

Otherwise, no, not really your fault. roc-wasm4 has a super dumb allocator. I assumed a dead simple one would be fine for how small wasm4 games would be, but it sounds like that was a bad assumption.

No concrete why glitching is happening. My best guess is that it is running out of memory but not correctly crashing. That or maybe it we are allocating too much memory for the heap in general. As such, the end of the heap is overlapping with the start of the screen or something like that.

[bhansconnect]

I probably should try to implement something like umm_malloc. Otherwise, a basic free list malloc may work.

[Walexander]

First question, is this built with optimization on? This may be a case where simply enabling optimization would fix the issue.

This particular profile is not using an optimized build but, with or without, the glitching remains.

That or maybe it we are allocating too much memory for the heap in general.

It does crash if I lower the mem-size but glitching doesn't seem to be affected regardless of the size.

As such, the end of the heap is overlapping with the start of the screen or something like that.

That would explain why the glitch starts at the bottom of the screen and grows upward.

[bhansconnect]

Actually, looking at the w4 source, the frame buffer is at the beginning of memory: pub const FRAMEBUFFER: *[6400]u8 = @ptrFromInt(0xA0);.

So I'm really not sure what is leading to the overwrites. But I do think it is somehow caused by high memory pressure. Just not sure how at the moment.

EDIT: definitely could also just be a roc compiler bug. If it is a known bug, I would guess it is caused by https://github.com/roc-lang/roc/issues/6434

[bhansconnect]

No idea if perf will be better or if it will fix the issues, but can you try with the umm_malloc branch?

[Walexander]

No idea if perf will be better or if it will fix the issues, but can you try with the umm_malloc branch?

Wow, the performance has improved significantly -- by an order of magnitude -- and no longer seeing roc_alloc even appear in the flame graph. 👍

The glitching remains, unfortunately but I wonder if it might be due to a partial frame being shown, instead of memory corruption.

I've attached a profile that shows the glitch at ~1.8s -- Immediately prior to the glitch there's a long update call but no frame is shown until after the next animation frame request.

Regardless, thank you so much! Glitches at 1860ms.json

I've

[bhansconnect]

Wow, the performance has improved significantly -- by an order of magnitude -- and no longer seeing roc_alloc even appear in the flame graph.

Awesome, will definitely make a PR for this.

The glitching remains, unfortunately but I wonder if it might be due to a partial frame being shown, instead of memory corruption.

Just for context, wasm4 has a single source of truth buffer in zi. It will write directly to the buffer whenever an effect to draw is made. Wasm4 also runs single threaded. So the actually display will not update until the entire update function is complete. Then it will copy from the frame buffer over to the real frame with a js draw command of some sort.

So it shouldn't be possible to get a render at a half update state, but idk.

[bhansconnect]

Do glitches only happen when quicksort is called? Like when you sort the the list referenced by the graph?

[Walexander]

Do glitches only happen when quicksort is called? Like when you sort the the list referenced by the graph?

Still glitching even after removing the List.sortWith calls.

[bhansconnect]

Ok, figured out the issue, not sure the correct fix though. You have a stack overflow. The stack grows downward and is right above the frame buffer.

I think part of the problem is current roc generation of task chains. One thing that may help is using Task.loop any time you have a list of tasks. (I think it compiles to a proper loop and reduces task chaining)

Also, fixing https://github.com/roc-lang/roc/issues/6434 probably would help. It blocks a lot of recursive function from being able to inline at all. As such, it leads to deeper stack traces.

It may also be safe just to up the stack size in build.zig, but I'm not really sure the ramifications.

This is definitely the type of issue that mostly doesn't affect anything on desktop, but matters a lot for embedded.

[bhansconnect]

Yeah, doubling the stack size and having the memory size seems to fix the issue in ReleaseSmall mode. There are still a couple of pixels that seem to glitch out, but that seems to be from a different cause (not sure what though).

[bhansconnect]

Also, really cool game! This is where it would really be nice if we had an unrestricted platform similar to wasm4. It would enable using all the simple drawing primitives, but would have much larger color and memory limits.

[bhansconnect]

This looks to fix the issue: https://github.com/roc-lang/roc/pull/6916

At least if you build in release.

[Walexander]

No more glitching when I build with the latest roc main branch!

@bhansconnect thank you!