gocode is using gigabytes and gigabytes of memory

[tekkamanendless]

Ever since I switched to the stamblerre fork to get Go 1.11's modules to work, gocode has been really, really slow, and it takes up gigabytes and gigabytes of RAM and ultimately gets OOM'ed. This seems to happen when Atom is trying to autocomplete as I'm typing, or sometimes when I've saved a file with invalid syntax. I essentially have a while loop running that kills gocode every 30 seconds to play it safe.

I'm really not sure what's up.

What version of Go are you using (go version)?

go version go1.11.4 linux/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOOS="linux"

What is the debug output of gocode?

Not particularly relevant, but:

2019/01/16 21:17:59 =======================================================
2019/01/16 21:17:59 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/types.go:230:8:undeclared name: str
2019/01/16 21:17:59 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:11:2:"github.com/davecgh/go-spew/spew" imported but not used
2019/01/16 21:17:59 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:4:2:"bytes" imported but not used
2019/01/16 21:17:59 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:5:2:"encoding/json" imported but not used
2019/01/16 21:17:59 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:6:2:"fmt" imported but not used
2019/01/16 21:17:59 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:7:2:"io/ioutil" imported but not used
2019/01/16 21:17:59 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:9:2:"net/url" imported but not used
2019/01/16 21:17:59 Elapsed duration: 7.867751923s
2019/01/16 21:17:59 Offset: 0
2019/01/16 21:17:59 Number of candidates found: 1
2019/01/16 21:17:59 Candidates are:
2019/01/16 21:17:59   type string string
2019/01/16 21:17:59 =======================================================
2019/01/16 21:18:00 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/types.go:230:8:undeclared name: st
2019/01/16 21:18:00 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:7:2:"io/ioutil" imported but not used
2019/01/16 21:18:00 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:9:2:"net/url" imported but not used
2019/01/16 21:18:00 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:11:2:"github.com/davecgh/go-spew/spew" imported but not used
2019/01/16 21:18:00 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:4:2:"bytes" imported but not used
2019/01/16 21:18:00 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:5:2:"encoding/json" imported but not used
2019/01/16 21:18:00 error in package bitbucket.org/tekkamanendless/emergencyreporting/erclient: /home/doug/src/bitbucket.org/tekkamanendless/emergencyreporting/erclient/client.go:6:2:"fmt" imported but not used
2019/01/16 21:18:00 Elapsed duration: 7.895922867s
2019/01/16 21:18:00 Offset: 0
2019/01/16 21:18:00 Number of candidates found: 1
2019/01/16 21:18:00 Candidates are:
2019/01/16 21:18:00   type string string
2019/01/16 21:18:00 =======================================================

What (client-side) flags are you passing into gocode?

Whatever go-plus is doing in Atom.

What editor are using?

Atom

[stamblerre]

I would expect gocode to be a little slower in modules, but I haven't heard any reports as drastic as this. Is it usable or is it so slow that you don't get completions? Is your project extremely large? To check that it's not related to Atom, do you mind testing it out on the command line? You can run gocode on the command line with gocode -in /path/to/file.go autocomplete /path/to/file.go <offset>.

[tekkamanendless]

The problem seems to appear in parallel contexts. I captured calls that Atom was making and ran them through gocode on the command line. Serially, there is no problem and everything works somewhat slowly, but as expected. In parallel (I ran 30 at a time for this test), gocode's memory usage ballooned to over 3GB. After completing all of the requests, it maintains that memory footprint.

My guess is that in the Atom use case, it's calling gocode more or less for every character that I type, and because the run time is longer due to the known caching deficiency in this fork, it runs into the memory-ballooning problem.

In some of my other Go projects, I know that I've had to manually call the garbage-collector after an individual task was completed because Go just wouldn't free up that memory for whatever reason. Maybe that's the case here.

Anyway, I can provide my exact code and the script to reproduce my memory issue. My project is just five files for a grand total of 45KB. I'll see if I can make a smaller reproducer for you.

[tekkamanendless]

Simple reproducer:

/tmp/reproducer/go.mod

module example.com/memory-issue

require github.com/davecgh/go-spew v1.1.1

/tmp/reproducer/main.go

package main

import (
        "fmt"

        "github.com/davecgh/go-spew/spew"
)

func main() {
        fmt.Printf("Hello, world!")
        spew.Dump("Some text")
}

type s1 struct {
        a string
}

Run this, and it uses up 0.5GB:

for (( i=0; i<10; i++ )); do $GOPATH/bin/gocode -in /tmp/reproducer/main.go autocomplete /tmp/reproducer/main.go 154 & done; wait;

Run this, and it doubles it to 1.0GB:

for (( i=0; i<20; i++ )); do $GOPATH/bin/gocode -in /tmp/reproducer/main.go autocomplete /tmp/reproducer/main.go 154 & done; wait;

Memory consumption seems proportional to the number of concurrent requests.

[stamblerre]

Thank you for the repro. This seems to be the issue mentioned in https://github.com/stamblerre/gocode/pull/22. I was hesitant to merge this change because of the additional dependency, but I will begin reviewing it.

[marcelino-m]

Same here, gocode is too slow in modules.

[stamblerre]

Just merged #22, so should be fixed.

[tekkamanendless]

Next up, I think that I'll need to patch go-plus for Atom to cancel requests faster (I don't think that it actually cancels requests after it sends the next ones).

[tekkamanendless]

Okay, short update: go-plus for Atom cancels its requests fairly quickly. I've patched gocode to call runtime.GC() after every request is handled, and while that helps, my gocode still balloons out to gigabytes and gigabytes (albeit much slower than before).

Is something still holding on to references to some of that stuff somewhere? I'll poke around with profiling it later.

[tekkamanendless]

Second update: calling debug.FreeOSMemory() instead of runtime.GC() after each request keeps gocode's memory from staying huge. It grows quickly while there are a bunch of requests in-flight, but it quickly drops back down to negligible amounts of memory immediately afterward.