Open danrobi11 opened 3 years ago
aartaka
commented
3 years ago Yes, that is fair. One place I'm sure we're not freeing the memory in is ffi-buffer-load -- WebKitBackForwardListItems are not freed, while they should.
There definitely are more places, so thanks for pointing that out!
UPD: No, it's not fii-buffer-load. It's something else. :thinking:
Ambrevar
commented
3 years ago I haven't looked at the code, but @aartaka any clue where this should be fixed? Is it a cl-webkit issue or a Nyxt issue?
jmercouris
commented
3 years ago Perhaps we have references left to the buffer in form of connected g-signals. Maybe those have to be disconnected for a buffer to be cleaned up?
aartaka
commented
3 years ago I haven't looked at the code, but @aartaka any clue where this should be fixed? Is it a cl-webkit issue or a Nyxt issue?
I looked over all the WebKit documentation regarding things we use in Nyxt, and I'm puzzled -- memory is mostly controlled by WebKit, so we don't need to free it. ¯_(ツ)_/¯
Ambrevar
commented
3 years ago Artyom Bologov notifications@github.com writes:
I haven't looked at the code, but @aartaka any clue where this should be fixed? Is it a cl-webkit issue or a Nyxt issue?
I looked over all the WebKit documentation regarding things we use in Nyxt, and I'm puzzled -- memory is mostly controlled by WebKit, so we don't need to free it. ¯_(ツ)_/¯
That's what I thought.
I remember discussing this issue in the past and testing to see if the memory was freed in Nyxt: it is, but not as much as we would like it to be.
So it seems that we are holding values in memory somewhere, somehow.
aartaka
commented
3 years ago One place we certainly don't clear memory in is cl-webkit's javascript-evaluation-complete callback. We need to unref the JSResult and free the result string after we move it to Lisp-land.
Given how much JS calls we do, these can take quite a lot of memory with time.
Ambrevar
commented
3 years ago I think I found another spot, but it's probably not much: when we create a window, its first active buffer is set to a dummy buffer. When switching to another buffer, we lose Lisp references to this buffer, but it's never `ffi-buffer-delete'd.
aartaka
commented
3 years ago JS objects are freed in https://github.com/joachifm/cl-webkit/commit/c554b02e29095e7739df12947526a03c8b34190d. How do we test the change? :thinking:
UPD: cc16e7e9e99e576b2c1dbb2228fadc25691ae394 incorporates this change into Nyxt, along with other niceties :)
Ambrevar
commented
3 years ago Thanks!
We need to come up with a systemic way to measure memory. Any idea how to do that?
Ambrevar
commented
3 years ago I don't know if Valgrind can be of any help for SBCL executable.
Maybe an SBCL extension or some Common Lisp library can help?
Worst case we can use GNU time to measure memory usage. It's a bit rough, but it's a start.
Then we need a recipe: any idea of a scenario that eats up memory fast?
danrobi11
commented
3 years ago I compiled version: Version dont-use-this-tag2 (weird name version 😉)
On my end the memory is flushed after deleting buffers. I'll leave to you to close the issue or not.
Ambrevar
commented
3 years ago I compiled version: Version dont-use-this-tag2 (weird name version 😉)
These are dummy tags I've introduce while playing with the pipeline. You can remove them. If you reclone the repo, these tags should be gone.
On my end the memory is flushed after deleting buffers. I'll leave to you to close the issue or not.
Thanks for reporting.
The issue is still present, but the problem is not just from buffer deletion. It might have to do with history.
Ambrevar
commented
3 years ago Something else I noticed is that we might not clean up deleted buffers properly, because we keep references to them with recent-bufers. This is intentional, but the web view should be cleaned up at least. We need to check that we do it.
It seems that this issue has been mitigated recently and it only affects few people. I'll remove the 2.0 tag.
Describe the bug Nyxt doesnt flush the memory after deleting buffers.
Precise recipe to reproduce the issue I've let nyxt run overnight. It built up memory. In the morning I deleted all buffers except for the
helpbuffer. Memory still remains the same. I'd have to close Nyxt to regain my memory.Information OS: MX 19.3 (patito feo) x86_64 based Debian GNU/Linux 10 (buster) SysV init version: 2.93 Kernel: 4.19.0-9-amd64 Xorg v1:7.7+19 WM: Spectrwm
System-information: Nyxt version: 2-pre-release-5 Operating system kernel: Linux 4.19.0-13-amd64 Lisp implementation: SBCL 2.0.10 Features: (WEBKIT2 WEBKIT2-2.30 WEBKIT2-TRACKING WEBKIT2-MUTE WEBKIT2-EMOJI WEBKIT2-MEDIA WEBKIT2-SANDBOXING GTK-3-22 GTK-3-20 GTK-3-18 GTK-3-16 GTK-3-14 GTK-3-12 GTK-3-10 GTK-3-8 GTK-3-6 GTK-3-4 GTK GDK-3-22 GDK-3-20 GDK-3-18 GDK-3-16 GDK-3-14 GDK-3-12 GDK-3-10 GDK-3-8 GDK-3-6 GDK-3-4 CAIRO-1-10 CAIRO-1-12 GDK-PIXBUF GLIB-2-30 GLIB-2-32 GLIB-2-34 GLIB-2-36 GLIB-2-38 GLIB-2-40 GLIB-2-42 GLIB-2-44 GLIB-2-46 GLIB-2-48 GLIB-2-50 GLIB-2-52 GLIB-2-54 GLIB-2-56 GLIB FSET-EXT-STRINGS SWANK PLUMP-UTF-32 GLOBAL-VARS DECLARE-TYPES PARENSCRIPT NAMED-READTABLES OSICAT-FD-STREAMS 21BIT-CHARS CHUNGA FLEXI-STREAMS CLOSER-MOP CL-PPCRE-UNICODE CL-UNICODE BORDEAUX-THREADS SPLIT-SEQUENCE GRAY-STREAMS FAST-IO-SV FAST-IO SBCL-USES-SB-ROTATE-BYTE ASDF-SYSTEM-CONNECTIONS CL-JSON-CLOS CL-JSON THREAD-SUPPORT CL-PPCRE FLAT-NAMESPACE X86-64 UNIX CFFI FLAT-NAMESPACE SEQUENCE-EMPTYP QUICKLISP ASDF3.3 ASDF3.2 ASDF3.1 ASDF3 ASDF2 ASDF OS-UNIX NON-BASE-CHARS-EXIST-P ASDF-UNICODE X86-64 GENCGC 64-BIT ANSI-CL COMMON-LISP ELF IEEE-FLOATING-POINT LINUX LITTLE-ENDIAN PACKAGE-LOCAL-NICKNAMES SB-CORE-COMPRESSION SB-LDB SB-PACKAGE-LOCKS SB-THREAD SB-UNICODE SBCL UNIX)
Output when started from a shell