Space leak when app state accumulates?

[eigengrau]

Haskell memory leaks still confuse me, so this behavior might be expected from my code, but I’ve noticed that when application state is carried over between events, the heap blows up very fast. E. g., for a simple app which has state based on Text, and which appends 1k of text when each event fires, the heap size goes up very fast and memory will soon run out, even though only a linear amount of data is appended. If I change the txt resp. str widgets to only take as many lines and columns as available, the effect is reduced, but still present (so maybe the leak is with Vty, not Brick?).

This came up when I was trying to use a str widget to keep a log of incoming Vty events. After a few hundred lines the address space exceeded several GB.

I can reproduce this behavior with the following gist.

The following gives the heap profile for the gist after hammering away at the keys for a second.

However, even though I should have library profiling enabled, neither the Vty or Brick modules appear in the graph, so it may well be it’s All My Fault™ after all. Maybe the issue makes more sense to you.

Thanks for making Brick!

[eigengrau]

I was able to get the profiler to take library functions into account as well. While I have library-profiling enabled in my global Cabal config, it seems I had to reinstall dependencies with --ghc-option=-fprof-auto -p.

The textual report can be found here.

COST CENTRE          MODULE                         %time %alloc
wcswidth             Graphics.Text.Width             50.6   42.6
str.theLines         Brick.Widgets.Internal          36.8   34.1
txt                  Brick.Widgets.Internal           7.4   19.2
iso10646String       Graphics.Vty.Image               2.0    3.1
unstreamChunks/inner Data.Text.Internal.Lazy.Fusion   1.3    0.0

[eigengrau]

Tentatively, @8e1d7f4 seems to address the issue. Heap Profiler output showing the retainer of allocations showed that renderFinal seemed to be keeping data from being garbage-collected. I don’t know whether this is the best way to address this, but forcing evaluation on the new render state allows the GC to kick in.

The new profile still shows a largish-seeming memory buildup in the str widget, but after forcing evaluation, the memory is successfully GC’ed.

I don’t understand brick well enough to give a well-founded account of why so much memory was retained there. My hypothesis is that when a widget contains computations that don’t actually go into the render (such as the str widget referencing more lines than can be shown on screen), those references are somehow kept around since the GHC run-time doesn’t know that after an image has been rendered, any left-over computations referenced within the rendering code-path aren’t needed anymore.

[eigengrau]

While the GC can now recycle unneeded computations at the point the image is rendered, str still builds up a considerable amount of thunks for very little input. 6330a59ff5c33860b982deb88ddbe2c718e63edb tries to cut this down by limiting line-rendering computations to only those parts of the string that can actually be rendered.

The str widget computations only appear under the category OTHER on the new heap profile.

This adds a dependency on deepseq, but it can probably be done by spraying bang patterns and seqs around.

[jtdaugherty]

Thank you for the report and the terrific investigation. I tend to hate dealing with space leaks so I especially appreciate that you have gone to the trouble to fix this. :)

[simonmichael]

+1, awesome issue report & fix.

[eigengrau]

Cheers! 😸