Memory leak in Browser runtime

[Skehmatics]

Description When using the Browser runtime, there appears to be a memory leak linked to lighting. With 4 lights in a scene, the Armory tab will consume a few MB every second until the system is out of memory. This appears to worsen as the shadow cube size increases.

Oddly, this increase in usage is not shown in the Firefox memory tools, but can be seen very clearly by watching its processes in a system monitor (Or waiting long enough for the system to OOM)

To Reproduce

1. Open blender
2. Add 3 additional lights to the default scene.
3. Change armory player > runtime to "Browser"
4. (optional) Add a new Render Path, set Shadows > Cube Size to the max (4096)

Expected behavior The same behavior when targeting Krom (no leak).

System Blender: 2.93.5 Armory: 2021.12 OS: Fedora 35 Graphics card: AMD RX 5700XT Browser: Firefox 94.0

Test File leakexample.zip

[MoritzBrueckner]

Can confirm this on Firefox 94.0 on Windows as well. After half a minute or so the Armory tab takes away roughly a GB of RAM.

I don't know yet what exactly the values inside the browser's performance profiler mean, but there seem to be a relative high amount of allocations in LightObject.updateClusters():

[MoritzBrueckner]

Some more findings:

This is the Dominators View that shows which objects can be reached from the garbage collector roots (= they aren't freed) for a time close to the start of the game:

and this is the same tree a bit later:

This seems to be the only big difference, you can see that there are a lot of small blocks of compiled code which appear to take away the memory. I wonder if these are closures that somehow can't be freed because they are still referenced somewhere? The entire sub-tree of the scripts > js::BaseScript GC root belongs to iron.object.Uniforms.setObjectConstant():

Maybe someone knows what could cause this? My above assumption of Kha's Image.lock() function seems to be wrong, the entire texture is allocated on each frame (which is still bad I guess), but I don't think that this causes the problems as there seem to be no references to the bytes object after updateClusters() goes out of scope.

---

If someone wants to play around with the captures (extract first, t1 is the first snapshot and t2 the second): snapshots.zip

You can import them as follows: "Speicher" means "Memory".

[MoritzBrueckner]

Some more observations on this issue:

• I took a closer look at the memory snapshots and now I think the leak is not related to any javascript objects. There are more JitCode entries in the second capture but they only create a difference of less than 1MB. The first capture consists of 11,99 MB memory captured and the second one is 12,16 MB – that's definitely not our GB large memory leak. Could it be related to WebGL?
• The memory leak does not happen if there is only one point light in the scene. As soon as you add another one the leaking will start. Maybe it is the above mentioned updateClusters() after all?
• The issue happens for both deferred and forward rendering.
• Post Process/Compositor/HDR is turned of and the leak still exists.

Btw, can anyone confirm this issue on a browser other than Firefox?

Edit: there is no memory leak if I only set the clustersData bytes in updateClusters() if the bytes variable is null and make it static (this is not a working fix of course as it probably breaks dynamic lighting, it's just to try what causes the issue). So, the issue seems to be somewhere along those lines: https://github.com/armory3d/iron/blob/f9bd0de649d47c92f680f0fe4b7c77a65d943391/Sources/iron/object/LightObject.hx#L425-L511.

Edit 2: seems that there is some issue with haxe.io.Bytes, can anyone confirm that running this example also causes a memory leak? https://try.haxe.org/#C924d15e. The bytes object should be garbage collected because it is set to null and also is only local to a scope so setting it to null shouldn't be required, but it still is not GC'd. Code from the try.haxe snippet:

class Test {
    static function main() {
        trace("Memory Leak test");

        haxe.MainLoop.add(mainloop);
    }

    static function mainloop() {
        var bytes = haxe.io.Bytes.alloc(100);
        trace('Allocated ${bytes.length} bytes');
        bytes = null;
    }
}

[tong]

@MoritzBrueckner side note: don't use trace in looping functions at tryhaxe. it falsifies memory consumption cause of the endless created log messages.

Without the trace, Bytes doesn't seem to leak memory.

[MoritzBrueckner]

@tong Thanks, I didn't know that. Indeed it seems that the leak goes away if I get rid of the traces. I originally used them to ensure that the compiler doesn't even think of removing the allocations (because they are unused, even though they constitute a side-effect), but adding up all the allocated lengths instead of printing them also causes no leak.

So we're one step back. Something with locking/unlocking the clusters data seems to leak memory and now I really have no idea why...