search

go-delve / delve

Delve is a debugger for the Go programming language.
MIT License
22.78k stars 2.14k forks source link

[feature request] coredump, show estimated memory usage / number of children objects recursively (object's total retained size) #2096

Open balta2ar opened 4 years ago

balta2ar commented 4 years ago

When it comes to memory leaks, heap profile is a useful tool to see where memory was allocated. However, that's only partly helpful because the profile shows where memory was allocated but it doesn't show where memory is currently referenced. coredump of a process is another view at the data, and delve helps greatly to look into it. The inconvenience is that if you're tracking memory leaks and you're unfamiliar with the code base, you can find leaking objects (e.g. slices or maps that are being appended to but never removed), but you need to know exactly where to look for. Alternatively, you have to check every goroutine, every stack frame of it, and all the local variables in each stack frame's local variable list, which can be tedious and time-consuming.

What I propose is to add some sort of summary to each object (including a goroutine, a stack frame, and a variable) that would tell the number of the child objects and total memory size, so that you can have an overview of the goroutnies/stack frames/and variables that take most of the memory and you could easily drill down into a goroutine -> stack frame -> leaking variable. Very similarly to how disk usage apps show this information, but for variables and memory here.

It was suggested that viewcore could be a tool for that (https://github.com/golang/go/issues/34561#issuecomment-536646220), but in practice delve shows type information much better than viewcore.

Ideally, this could be a part of gdlv UI, but it could also work as a starlark command. I'm not sure, maybe what I want it possible using Variable API and this example https://github.com/go-delve/delve/blob/master/Documentation/cli/starlark.md#print-all-elements-of-a-linked-list?

aarzilli commented 4 years ago

Using the preexisting variable reading code for this would be prohibitively expensive for any heap of realistic size. You would have to integrate viewcore (or reimplement it) inside delve (not necessarily a bad idea). But I don't know why delve's type information is better than viewcore.

balta2ar commented 4 years ago

why delve's type information is better than viewcore

I don't know how both of them work, but delve shows correct types everywhere, while viewcore shows a lot of "unk8", "unk64" etc. Could be my negligence or omission though.

I don't know why it's not a part of the viewcore CLI API, but they have an implementation of an algorithm to calculate dominators: https://github.com/golang/debug/blob/master/internal/gocore/dominator.go, implemented in 2018 (https://github.com/golang/debug/commit/545543929d8ccc043f8e537de66eafa69c90e046), which is claimed to be at least as good as O(nlog(n)). Having tackled it only a bit so that its output is available as a viewcore command, right now I don't fully understand how to correctly use its output, but maybe later I will or it could be a starting point for someone else.