Open ZSA233 opened 5 months ago
andydotxyz
commented
5 months ago Out of interest can you give it half a chance? Wait until the window is visible and also refresh at less than your screens refresh rate? For example a 100ms delay and then a 20ms ticker? I don't know if it will be resolved with that but it may be different as the renderers will be properly initialised once drawn.
ZSA233
commented
5 months ago Out of interest can you give it half a chance? Wait until the window is visible and also refresh at less than your screens refresh rate? For example a 100ms delay and then a 20ms ticker? I don't know if it will be resolved with that but it may be different as the renderers will be properly initialised once drawn.
The testing code just a easy and simple way to reproduce the problem, not a regular scene. Actually I'm using it a very low refresh rate in production environment, such as 2 seconds refresh 4 * 100 cells, but it still take 100mb ram incr every 3 hours.
dweymouth
commented
5 months ago I think the table code could possibly be improved (looking quickly at the code I don't understand why we need to create a new template cell every refresh) but I don't believe there's a memory leak here. The renderer cache is periodically cleaned out to remove expired renderers for widgets that are not being drawn anymore, so while this will cause memory to go up for a bit, the renderers will eventually be GC'ed. What you may be seeing is Go holding on to more memory than is actually in use by the app too in case it needs to allocate more. Now running refresh every 10 ms will cause a lot of memory to accumulate because IIRC the caches are cleaned every 1 or 2 minutes.
dweymouth
commented
5 months ago Another thing @ZSA233 if you're updating a table and refreshing for each cell (not sure if you're doing this) - it's better to update all the cells and then refresh the table once. (That is assuming you update all the cells together). In general it's important for performance in Fyne to refresh only when needed, and at the lowest refresh scope too. Basically don't refresh until you want the widget to be redrawn.
ZSA233
commented
5 months ago I think the table code could possibly be improved (looking quickly at the code I don't understand why we need to create a new template cell every refresh) but I don't believe there's a memory leak here. The renderer cache is periodically cleaned out to remove expired renderers for widgets that are not being drawn anymore, so while this will cause memory to go up for a bit, the renderers will eventually be GC'ed. What you may be seeing is Go holding on to more memory than is actually in use by the app too in case it needs to allocate more. Now running refresh every 10 ms will cause a lot of memory to accumulate because IIRC the caches are cleaned every 1 or 2 minutes.
You are right for the cache cleaning code in the driver, so cache.Renderer() may not be the reason for the memory leak. However, the leak still exists for somewhere, as it persists even after several hours of executing table.Refresh(), even at a low refresh rate.
Showing nodes accounting for 233.01MB, 91.87% of 253.63MB total
Dropped 121 nodes (cum <= 1.27MB)
Showing top 20 nodes out of 143
flat flat% sum% cum cum%
35.51MB 14.00% 14.00% 52.51MB 20.70% fyne.io/fyne/v2/widget.(*RichText).cachedSegmentVisual
28.72MB 11.32% 25.32% 163.59MB 64.50% fyne.io/fyne/v2/internal/cache.Renderer
26.61MB 10.49% 35.81% 26.61MB 10.49% github.com/go-text/typesetting/opentype/loader.(*Loader).findTableBuffer
26.01MB 10.25% 46.07% 28.01MB 11.04% fyne.io/fyne/v2/widget.NewLabelWithStyle
18MB 7.10% 53.17% 18MB 7.10% fyne.io/fyne/v2/widget.NewRichText (inline)
16.50MB 6.51% 59.67% 16.50MB 6.51% fyne.io/fyne/v2/canvas.NewText (inline)
12.51MB 4.93% 64.60% 12.51MB 4.93% github.com/go-text/typesetting/opentype/tables.(*SimpleGlyph).parsePoints
12MB 4.73% 69.34% 12MB 4.73% fyne.io/fyne/v2/canvas.NewRectangle (inline)
11MB 4.34% 73.67% 11MB 4.34% runtime.malg
10MB 3.94% 77.62% 28MB 11.04% fyne.io/fyne/v2/widget.NewRichTextWithText
5.50MB 2.17% 79.79% 19.01MB 7.49% github.com/go-text/typesetting/opentype/tables.(*Glyph).parseData
4.50MB 1.77% 81.56% 4.50MB 1.77% time.NewTicker
4.50MB 1.77% 83.33% 4.50MB 1.77% fyne.io/fyne/v2/internal/cache.(*expiringCache).setAlive
4.15MB 1.64% 84.97% 4.15MB 1.64% github.com/riobard/go-bloom.New
3.50MB 1.38% 86.35% 15.21MB 6.00% fyne.io/fyne/v2/widget.(*RichText).updateRowBounds.func1
3.50MB 1.38% 87.73% 3.50MB 1.38% fyne.io/fyne/v2/internal/widget.NewSimpleRenderer (inline)
3MB 1.18% 88.91% 76.01MB 29.97% fyne.io/fyne/v2/widget.(*RichText).CreateRenderer
3MB 1.18% 90.10% 3MB 1.18% fyne.io/fyne/v2/widget.(*BaseWidget).ExtendBaseWidget
2.50MB 0.99% 91.08% 55.51MB 21.89% fyne.io/fyne/v2/widget.(*textRenderer).Refresh
2MB 0.79% 91.87% 2MB 0.79% fyne.io/fyne/v2/widget.splitLines253.63MB total, but it actually takes up almost 700MB, meaning the leak may not be in Go structs, but could be at the cgo level.andydotxyz
commented
5 months ago This could be a duplicate of text cache issues we are working on, if the text in the cells is changing on each refresh. I was perhaps diverted from thinking this based on the assertion that it was the create calls being over used that was the source of the issue. On further investigation does it seem like the memory rather than the function calls is the issue being investigated?
dweymouth
commented
5 months ago Could be, though I still don't think it's an actual memory leak, as the text texture cache is cleaned out on a schedule too. It's an issue of more memory being consumed than needed though since each unique line of text (as opposed to glyph) shown is a unique entry in the texture cache.
andydotxyz
commented
5 months ago It shouldn't grow like the OP is reporting though surely - over hours of usage?
ZSA233
commented
5 months ago It shouldn't grow like the OP is reporting though surely - over hours of usage?
Yeap, the OP's testing code may be incorrect for reproducing my prod env problem. I have been re-testing the OP code for over an hour, the memory usage has been stable, running between 100~200MB, which is not a significant issue. Therefore, I make another test that is closer to the memory issues in prod env:
PS C:\Users\ZSA> go tool pprof http://127.0.0.1:9996/debug/pprof/heap
Fetching profile over HTTP from http://127.0.0.1:9996/debug/pprof/heap
Saved profile in C:\Users\ZSA\pprof\pprof.___6go_build_test.exe.alloc_objects.alloc_space.inuse_objects.inuse_space.001.pb.gz
File: ___6go_build_test.exe
Build ID: C:\Users\ZSA\AppData\Local\JetBrains\GoLand2024.1\tmp\GoLand\___6go_build_test.exe2024-06-07 00:59:15.714434 +0800 CST
Type: inuse_space
Time: Jun 7, 2024 at 1:03am (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top20
Showing nodes accounting for 23784.30kB, 100% of 23784.30kB total
Showing top 20 nodes out of 131
flat flat% sum% cum cum%
9605.13kB 40.38% 40.38% 9605.13kB 40.38% github.com/go-text/typesetting/opentype/loader.(*Loader).findTableBuffer
2560.70kB 10.77% 51.15% 2560.70kB 10.77% fyne.io/fyne/v2/widget.(*RichText).cachedSegmentVisual
2272.37kB 9.55% 60.70% 2272.37kB 9.55% fyne.io/fyne/v2/internal/cache.SetFontMetrics
1056.33kB 4.44% 65.15% 1056.33kB 4.44% github.com/go-text/typesetting/opentype/tables.ParseBaseArray
1024.20kB 4.31% 69.45% 1024.20kB 4.31% fyne.io/fyne/v2/widget.NewLabelWithStyle
1024.19kB 4.31% 73.76% 8044.71kB 33.82% fyne.io/fyne/v2/widget.NewRichText (inline)
1024.06kB 4.31% 78.06% 2048.21kB 8.61% github.com/go-text/typesetting/opentype/tables.(*Glyph).parseData
596.16kB 2.51% 80.57% 596.16kB 2.51% github.com/go-text/typesetting/opentype/api/font/cff.parseIndexContent
522.70kB 2.20% 82.77% 522.70kB 2.20% github.com/go-text/typesetting/unicodedata.map.init.0
513.50kB 2.16% 84.93% 513.50kB 2.16% image.NewRGBA
512.44kB 2.15% 87.08% 512.44kB 2.15% github.com/go-text/typesetting/harfbuzz.newShaperOpentype
512.17kB 2.15% 89.24% 512.17kB 2.15% github.com/go-text/typesetting/opentype/api/font.(*Face).getPointsForGlyph
512.12kB 2.15% 91.39% 512.12kB 2.15% github.com/go-text/typesetting/harfbuzz.(*otMap).addLookups
512.10kB 2.15% 93.54% 512.10kB 2.15% github.com/go-text/typesetting/opentype/tables.(*SimpleGlyph).parsePoints
512.05kB 2.15% 95.69% 512.05kB 2.15% regexp/syntax.(*parser).newRegexp
512.05kB 2.15% 97.85% 512.05kB 2.15% github.com/go-text/typesetting/opentype/tables.(*CompositeGlyph).parseGlyphs 512.02kB 2.15% 100% 512.02kB 2.15% fyne.io/fyne/v2/internal/cache.SetCanvasForObject
0 0% 100% 15578.21kB 65.50% fyne.io/fyne/v2.MeasureText
0 0% 100% 16890.72kB 71.02% fyne.io/fyne/v2/internal/cache.Renderer
0 0% 100% 2562.25kB 10.77% fyne.io/fyne/v2/internal/driver.WalkVisibleObjectTree
(pprof)
PS C:\Users\ZSA> tasklist /FI "PID eq 11876"映像名称 PID 会话名 会话# 内存使用 ========================= ======== ================ =========== ============ ___6go_build_test.exe 11876 Console 1 155,352 K
2. Time: Jun 7, 2024 at 11:56am (CST)
```shell
PS C:\Users\ZSA> go tool pprof http://127.0.0.1:9996/debug/pprof/heap
Fetching profile over HTTP from http://127.0.0.1:9996/debug/pprof/heap
Saved profile in C:\Users\ZSA\pprof\pprof.___6go_build_test.exe.alloc_objects.alloc_space.inuse_objects.inuse_space.005.pb.gz
File: ___6go_build_test.exe
Build ID: C:\Users\ZSA\AppData\Local\JetBrains\GoLand2024.1\tmp\GoLand\___6go_build_test.exe2024-06-07 00:59:15.714434 +0800 CST
Type: inuse_space
Time: Jun 7, 2024 at 11:56am (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top20
Showing nodes accounting for 440.77MB, 96.11% of 458.63MB total
Dropped 86 nodes (cum <= 2.29MB)
Showing top 20 nodes out of 82
flat flat% sum% cum cum%
135.54MB 29.55% 29.55% 173.04MB 37.73% fyne.io/fyne/v2/widget.(*RichText).cachedSegmentVisual
69.51MB 15.16% 44.71% 69.51MB 15.16% fyne.io/fyne/v2/widget.NewLabelWithStyle (inline)
55.01MB 11.99% 56.70% 61.87MB 13.49% fyne.io/fyne/v2/widget.NewRichText (inline)
36MB 7.85% 64.55% 36MB 7.85% fyne.io/fyne/v2/canvas.NewText
32MB 6.98% 71.53% 93.87MB 20.47% fyne.io/fyne/v2/widget.NewRichTextWithText
25MB 5.45% 76.98% 25MB 5.45% fyne.io/fyne/v2/canvas.NewRectangle (inline)
22.50MB 4.91% 81.89% 43.71MB 9.53% fyne.io/fyne/v2/widget.(*RichText).updateRowBounds.func1
21.81MB 4.76% 86.65% 370.87MB 80.86% fyne.io/fyne/v2/internal/cache.Renderer
10.50MB 2.29% 88.94% 250.69MB 54.66% fyne.io/fyne/v2/widget.(*RichText).CreateRenderer
9.38MB 2.05% 90.98% 9.38MB 2.05% github.com/go-text/typesetting/opentype/loader.(*Loader).findTableBuffer
6MB 1.31% 92.29% 6MB 1.31% fyne.io/fyne/v2/widget.splitLines
5.50MB 1.20% 93.49% 181.54MB 39.58% fyne.io/fyne/v2/widget.(*textRenderer).Refresh
4.50MB 0.98% 94.47% 4.50MB 0.98% fyne.io/fyne/v2/internal/cache.(*expiringCache).setAlive
4MB 0.87% 95.34% 4MB 0.87% fyne.io/fyne/v2/theme.darkPaletColorNamed
3MB 0.65% 96.00% 4.50MB 0.98% github.com/go-text/typesetting/harfbuzz.NewFont
0.50MB 0.11% 96.11% 8.02MB 1.75% fyne.io/fyne/v2/internal/driver/glfw.(*gLDriver).startDrawThread.func1
0 0% 96.11% 15.21MB 3.32% fyne.io/fyne/v2.MeasureText
0 0% 96.11% 13.52MB 2.95% fyne.io/fyne/v2/internal/driver.WalkVisibleObjectTree
0 0% 96.11% 13.52MB 2.95% fyne.io/fyne/v2/internal/driver.walkObjectTree
0 0% 96.11% 13.52MB 2.95% fyne.io/fyne/v2/internal/driver.walkObjectTree.func1 (inline)
(pprof)
PS C:\Users\ZSA> tasklist /FI "PID eq 11876"
映像名称 PID 会话名 会话# 内存使用
========================= ======== ================ =========== ============
___6go_build_test.exe 11876 Console 1 1,009,812 Kmain.go
package main
import (
"fmt"
"fyne.io/fyne/v2"
"fyne.io/fyne/v2/app"
"strconv"
"time"
)
var rows = [][]string{}
func main() {
myApp := app.New()
myWindow := myApp.NewWindow("test")
myWindow.Resize(fyne.NewSize(1000, 500))
headers := []string{}
for i := 0; i < 100; i++ {
rows = append(rows, make([]string, 100))
headers = append(headers, "#"+strconv.Itoa(i))
for j := 0; j < 100; j++ {
rows[i][j] = fmt.Sprintf("%d:%d", i, j)
}
}
table := NewTableEx(headers)
myWindow.SetContent(table)
go runPprof()
go refreshLoop(table)
myWindow.ShowAndRun()
}
func refreshLoop(table *TableEx) {
tick := time.NewTicker(time.Millisecond * 500)
defer tick.Stop()
for _ = range tick.C {
for i := 0; i < 100; i++ {
for j := 0; j < 100; j++ {
rows[i][j] = fmt.Sprintf("%d:%d", time.Now().Second()+i, time.Now().Second()+j)
}
}
table.SetMatrix(rows)
}
}package main
import (
"fyne.io/fyne/v2"
"fyne.io/fyne/v2/widget"
"strconv"
"sync/atomic"
)
type TableEx struct {
*widget.Table
header atomic.Pointer[[]string]
rows atomic.Pointer[[][]string]
}
func (w *TableEx) Resize(size fyne.Size) {
w.Table.Resize(size)
}
func (w *TableEx) SetMatrix(ss [][]string) {
w.rows.Store(&ss)
w.Table.Refresh()
}
func NewTableEx(header []string) *TableEx {
table := widget.NewTableWithHeaders(nil, nil, nil)
table.ShowHeaderRow = true
table.ShowHeaderColumn = true
var hlp *TableEx
hlp = &TableEx{
header: atomic.Pointer[[]string]{},
rows: atomic.Pointer[[][]string]{},
Table: table,
}
hlp.header.Store(&header)
hlp.rows.Store(&[][]string{})
table.Length = func() (rows int, cols int) {
return len(*hlp.rows.Load()), len(*hlp.header.Load())
}
table.CreateCell = func() fyne.CanvasObject {
return widget.NewLabelWithStyle("", fyne.TextAlignCenter, fyne.TextStyle{})
}
table.CreateHeader = func() fyne.CanvasObject {
return widget.NewLabelWithStyle("", fyne.TextAlignCenter, fyne.TextStyle{
Bold: true,
})
}
table.UpdateCell = func(id widget.TableCellID, template fyne.CanvasObject) {
l := template.(*widget.Label)
var text string
rows := *hlp.rows.Load()
if id.Row < len(rows) && len(rows[id.Row]) > id.Col {
text = rows[id.Row][id.Col]
}
l.SetText(text)
}
table.UpdateHeader = func(id widget.TableCellID, template fyne.CanvasObject) {
l := template.(*widget.Label)
if id.Row < 0 {
h := *hlp.header.Load()
if id.Col >= len(h) {
} else {
l.SetText(h[id.Col])
}
} else if id.Col < 0 {
l.SetText(strconv.Itoa(id.Row + 1))
if l.Alignment != fyne.TextAlignTrailing {
l.Alignment = fyne.TextAlignTrailing
}
} else {
return
}
}
return hlp
}
package main
import (
"net/http"
"net/http/pprof"
)
func runPprof() {
//http.HandleFunc("/debug/pprof/", pprof.Index)
//http.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
//http.HandleFunc("/debug/pprof/profile", pprof.Profile)
//http.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
//http.HandleFunc("/debug/pprof/trace", pprof.Trace)
http.HandleFunc("/debug/pprof/allocs", pprof.Handler("allocs").ServeHTTP)
http.HandleFunc("/debug/pprof/block", pprof.Handler("block").ServeHTTP)
http.HandleFunc("/debug/pprof/goroutine", pprof.Handler("goroutine").ServeHTTP)
http.HandleFunc("/debug/pprof/heap", pprof.Handler("heap").ServeHTTP)
http.HandleFunc("/debug/pprof/mutex", pprof.Handler("mutex").ServeHTTP)
http.HandleFunc("/debug/pprof/threadcreate", pprof.Handler("threadcreate").ServeHTTP)
http.ListenAndServe("0.0.0.0:9996", nil)
}
If there is indeed a leak, I imagine it must be in GPU textures, as I'm less familiar with how that part of the codebase works and I'm pretty sure there's no leak on the widget/renderer side of things and awhile back did a bit of a codebase audit looking for leaks, fixed a few, but didn't find any in this area.
Or else it's Go holding on to memory because the OS isn't under memory pressure and asking for it back. AFAIK, the Go runtime marks freed memory as "unneeded" on most OSes but retains it, or at least a chunk of it, unless the OS asks for it back, to speed future allocations.
Jacalz
commented
5 months ago Or else it's Go holding on to memory because the OS isn't under memory pressure and asking for it back. AFAIK, the Go runtime marks freed memory as "unneeded" on most OSes but retains it, or at least a chunk of it, unless the OS asks for it back, to speed future allocations.
I’m no pprof magician but it feels like the built-in profiling in Go should report actual me memory usage and not how much it is holding on to? My best bet is that this is an actual leak somewhere.
andydotxyz
commented
5 months ago From what I can see TableEx is not extending correctly.
type TableEx struct {
*widget.Tableshould be:
type TableEx struct {
widget.Tableand there is no call to ExtendBaseWidget.
If the code will not produce the same leak with the regular widget.Table then I would say this may not be a leak in the toolkit directly.
Keima-lpj
commented
2 months ago I'm having a similar problem, once per second refresh the table and the memory will grow fast, about 1M a second. after some time passes, a gc will be triggered and the memory will be reduced from 200M to 50M. then it will grow slowly again. Much slower than the initial growth.
first time:
the other time:
andydotxyz
commented
2 months ago We cache all the graphical textures, which includes text rendered. When nothing has been shown the cache will grow fast. Then we clean out unused textures (approx each minute) and re-used items will stay in memory so growth later on will indeed be slower.
I'm not sure that this indicates a memory leak as per the original post.
knusbaum
commented
2 months ago @andydotxyz I ran into this issue, and there does appear to be a leak of sorts.
In the current glfw driver, the cache never gets cleaned. I looked at the mobile driver and did more or less what it was doing and was able to fix my issue, and also fix the issue in the testcase code in https://github.com/fyne-io/fyne/issues/4903#issuecomment-2153844849
My PR: https://github.com/fyne-io/fyne/pull/5112
@ZSA233 are you able to see if this fixes your issue?
As a side note, I noticed two things:
Checklist
Describe the bug
widget.Table will CreateCell every time when table.Refresh() called. detail below
How to reproduceHow to reproduce
testing code
}
func refreshLoop(table widget.Table) { tick := time.NewTicker(time.Millisecond 10) defer tick.Stop() for _ = range tick.C { table.Refresh() } }
Screenshots
No response
Example code
see
how to reproduce.Fyne version
2.4.5
Go compiler version
1.22.3
Operating system and version
macOS 13.1
Additional Information
No response