Memory leak in correlation with window resize

[madzione]

Memory gradually increases when resizing the window and the allocated memory never goes down again.

OS: Windows 10 64bit Go: 1.12.1 Fyne driver: OpenGL

[andydotxyz]

I expect this is a duplicate of #209 - but we will check once it has been resolved.

[andydotxyz]

I think a few leak fixes that went in today have helped. But probably just cut the problem in half rather than actually resolving it...

The search continues.

[ohir]

Confirmed as of today's update. This leak makes fyne unusable even for etp preview: after 5s of canvas resizes it makes to 5/2GB

    VIRT    RES    SHR %MEM COMMAND
 8467836 4.841G 1.923G 7.77 ./fyne_demo

linux 4.4.178, libglfw3-3.1.2

[andydotxyz]

Thanks for the update. In what way does it seem unusable? Is it too slow or does it crash or otherwise not work?

I’m trying to read the numbers you provided but I want to double check, is 4.8GB really just 7% of your RAM? (62GB total?)

[ohir]

@andydotxyz

In what way does it seem unusable?

One can not tell her users 'just do not resize your app'. And certainly it will not pass the Early Technology Preview evaluation that is usually done by mean, nitpicking gremlins who are eager to keep status-quo ;)

(~5GiB is ~7.7% of 64GiB, seems so. Line posted is pasted partially from ps v partially from top).

[andydotxyz]

I had not intended to say this is not a problem - it is. I was trying to understand better how the apps were not working for you. I perhaps understand “unusable” differently than you indented - apologies if that is the case.

In terms of memory in Go we have the fun of GC and various complications that we need to work out. If you stop resizing does the number go back down? I cannot replicate these massive numbers so I am trying to see what is different with your computer. Is it possible that with massive amounts of free memory the GC is not kicking in?

Are you resizing slowly or fast? Is it constant or start-stop? Do you see glitches in the rendering as you resize?

I’d love to fix this and the more information the better. Thanks.

[ohir]

@andydotxyz

how the apps were not working for you.

It was working. But it allocated huge amount of memory per every resizing move. After 5s it hold to 5GB RES. I raised this because I respect your work and I wish Fyne the best.

Are you resizing slowly or fast? Is it constant or start-stop? Do you see glitches in the rendering as you resize?

Fast: it was after around 5s of fast resizes. Constant, ie. grab a handle then move back and forth. No glitches no stutter (but it is a 4c i7 3GHz box).

If you stop resizing does the number go back down?

No. Your hunch that GC is not pressured is probably right but it is mostly irrelevant here.

In linux you can not really "return" memory to the OS, you only can flag it as unneeded. Therefore, without full profiling, one may not even guess how much memory a program is really holding to. At least not using standard OS tools.

Even if after a while the memory is available to the OS again, once allocated this peak is then shown to the user as perpetual usage till some other program will pressure OS enough to finally reclaim memory from the Go runtime. This can only be amplified with today's just suspend habits where machine is restarted only after kernel updates.

This is not that much of a problem with server applications — as people deploying servers are supposed to understand this. But average desktop users will not understand. For them, the impression of memory being gobbled will be a big NO and this will in turn be a huge problem of adoption for an app using Fyne.

[andydotxyz]

Thanks I understand your point. With regards to your point about server deployments understanding memory usage: on the desktop I don't know what percentage of people monitor available memory unless something stops working.

On a system with far less available RAM this is harder to debug as it does not manifest in the same way. Is there any way that you could set up a profiler to show the memory hotspots / main leak areas? That would be the most helpful thing to resolving this. Thanks in advance.

[asciifaceman]

I am unable to reproduce running fyne develop source on OSX 10.12 & go1.11.2 (just been lazy updating) right now, and I've been profiling around a bit with an absolutely bare minimum app.

main.go

func main() {
    f, err := os.Create("resize.prof")
    if err != nil {
        log.Fatal("could not create memory profile: ", err)
    }
    defer f.Close()
    runtime.GC() // get up-to-date statistics

    fyneapp := app.New()
    w := fyneapp.NewWindow("")
    w.SetContent(widget.NewVBox(
        widget.NewButton("Quit", func() {
            if err := pprof.WriteHeapProfile(f); err != nil {
                log.Fatal("could not write memory profile: ", err)
            }
            fyneapp.Quit()
        }),
    ))
    w.ShowAndRun()
}

note: must be exited with quit button to write profile

Profiling

go tool pprof -http=:8080 resize.prof

After about 30s of rapid resizing, I've seen no real impact on memory allocations.

I built a binary, loaded it up, and resized violently for a few minutes.

I'm attaching the profile, and a screenshot from after the resizing flurry.

resize.prof.zip

I unfortunately don't have any windows machines or UI-capable linux machines (headless only etc)

---

It also reported only 6 go routines on exit.

I also tried automating the process by setting up a process that automatically resized the window every millisecond (grow/shrinking) and the biggest memory consumer was my own scaffolding to set up that process.

---

Not meant to discount OP, just unable to reproduce on my end and system. I provided my test case so hopefully others can try as well.

[madzione]

Just to make it clear, the main issue(huge memory jumps) seems to be resolved since 5c71783, 9d63794 and 18ef6b4. Nevertheless there is still a small increase, but much slower.

[andydotxyz]

@ohir are you able to test the latest master or v1.0.1 and comment on whether this still occurs for you?

[ohir]

@andydotxyz. Tested with today's master. Now on more typical machine (i5/16GB). Looks better so far, ie. I had to keep resizing for waaay longer (around 3min) to get to 3GB limit I set.

First loose test: after some thousand resizes in various parts of demo (canvas, icons, widgets) I'd got:

ps:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
mgt7     17461 14.4 20.3 6338436 3311760 pts/15 Sl+ 13:28   0:46 ./fyne_demo

top:
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
17461 mgt7      20   0 6338436 3,159g  25200 S 1,333 20,33   0:52.07 fyne_demo

Next I did short 15s test of main demo window using ./fyne_demo & \n top -p $!

MAIN RESULTS:

Open main ps:
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
mgt7     17686  1.6  0.2 481280 35948 pts/15   Sl+  13:59   0:00 ./fyne_demo

Open main top:
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
17686 mgt7      20   0  481280  35948  15104 S 1,661 0,221   0:01.92 fyne_demo

After main idled 5min:
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
17686 mgt7      20   0  555652  35948  15104 S 0,333 0,221   0:14.10 fyne_demo

After main resizing for 15s:
  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
17686 mgt7      20   0 1505212 563676  16744 S 1,333 3,459   0:22.62 fyne_demo

Test 2: Reopen fresh main for each subtest. Use top. Resize each subpart for ~15s (circa 200-300 resizes) Init top usage is ~as above. Usages after 15s test are:

SUBPART RESULTS:

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
Canvas 
19498 mgt7      20   0 1798820 612400  16968 S 1,000 3,758   0:08.17 fyne_demo

Icons
19510 mgt7      20   0 1297816 647496  50560 S 1,333 3,974   0:18.12 fyne_demo

Layout:Border
19555 mgt7      20   0 1727412 247256  15904 S 1,667 1,517   0:02.71 fyne_demo

Layout:Box
19626 mgt7      20   0 1390396 170584  15664 S 2,658 1,047   0:02.73 fyne_demo

Layout:FixedGrid
19674 mgt7      20   0 1458480 161196  15920 S 2,000 0,989   0:02.87 fyne_demo

Layout:Grid
19661 mgt7      20   0 1594452 232472  16052 S 1,661 1,427   0:03.95 fyne_demo

Widgets:Buttons
19686 mgt7      20   0 1469468 235560  16048 S 1,333 1,446   0:03.59 fyne_demo

Widgets:Input
19699 mgt7      20   0 1532988 260424  16468 S 2,000 1,598   0:03.92 fyne_demo

Widgets:Progress
19731 mgt7      20   0 1675812 221104  15756 S 6,645 1,357   0:04.61 fyne_demo

Widgets:Scroll
19744 mgt7      20   0 1529660 483540  16692 S 1,333 2,968   0:05.49 fyne_demo

Form:
19756 mgt7      20   0 2544700 440328  16564 S 2,000 2,702   0:05.06 fyne_demo

Dialogs:
19816 mgt7      20   0 1744884 234556  15680 S 0,997 1,440   0:03.12 fyne_demo

Input:
19829 mgt7      20   0 1456420 229436  16164 S 2,326 1,408   0:04.61 fyne_demo

Advanced:
19841 mgt7      20   0 2338564 287204  16288 S 1,667 1,763   0:03.09 fyne_demo

P.S. I sincerely apologize for delays, but I still have other duties

[andydotxyz]

Thanks for the feedback here it's very helpful. We suspect that the remaining leaks relate to caches not clearing. This will be resolved in a driver improvement for 1.2 so I am moving this bug to that release.

[andydotxyz]

I'm pretty confident that this is no longer an issue - at least I cannot replicate. Please re-open if you have new information against the develop branch.