search

golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
123.83k stars 17.65k forks source link

cmd/compile: slow compile/build compared to Go 1.4 #13646

Closed kokizzu closed 7 years ago

kokizzu commented 8 years ago

It's been a while, we still use 1.4 because of this issue, it affect to much to our build-test-debug cycle.

$ uname -a
Linux f9988 4.1.14-1-lts #1 SMP Thu Dec 10 21:34:50 CET 2015 x86_64 GNU/Linux

# go 1.1 and go 1.2 failed to compile github.com/yosuke-furukawa/json5/encoding/json5

$ rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build
go version go1.3.3 linux/amd64

CPU: 5.35s      Real: 3.34s     RAM: 287316KB
CPU: 5.63s      Real: 3.40s     RAM: 287316KB
CPU: 5.87s      Real: 3.41s     RAM: 287272KB

$ rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build
go version go1.4.2 linux/amd64

CPU: 5.83s      Real: 3.34s     RAM: 290024KB
CPU: 5.71s      Real: 3.45s     RAM: 290024KB
CPU: 5.96s      Real: 3.46s     RAM: 289948KB

$ rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build
go version go1.5.2 linux/amd64

CPU: 21.93s     Real: 7.92s     RAM: 310148KB
CPU: 21.07s     Real: 7.46s     RAM: 318660KB
CPU: 21.45s     Real: 7.58s     RAM: 310160KB

packages from: http://seblu.net/a/archive/packages/g/go/

for comparison, it's now as slow as gcc-go:

# for comparison:
$ rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build
go version go1.4.2 gccgo (GCC) 5.3.0 linux/amd64

CPU: 20.76s     Real: 9.65s     RAM: 203212KB
CPU: 21.33s     Real: 9.83s     RAM: 203340KB
CPU: 21.09s     Real: 9.85s     RAM: 203336KB
cespare commented 8 years ago

What about after you've installed dependencies? (go build -i or go install)

kokizzu commented 8 years ago

it's significantly faster if not removing the pkg and doing go build -i at the first time..

1.5.2
CPU: 3.03s      Real: 1.99s     RAM: 243676KB
CPU: 3.04s      Real: 1.96s     RAM: 238416KB
CPU: 3.04s      Real: 1.96s     RAM: 238348KB

but still far slower than

1.4.2
CPU: 0.88s      Real: 0.99s     RAM: 207080KB
CPU: 0.93s      Real: 0.99s     RAM: 207000KB
CPU: 0.87s      Real: 1.00s     RAM: 207084KB
ALTree commented 8 years ago

Isn't a 2x slowdown between 1.4 and 1.5 expected? See https://golang.org/doc/go1.5#performance

Builds in Go 1.5 will be slower by a factor of about two

kokizzu commented 8 years ago

it got lot worse on windows 8,1 64-bit, with slower processor but SSD harddisk

1.4.3

$ time go build

real 0m7.670s user 0m0.015s sys 0m0.000s

real 0m6.886s user 0m0.000s sys 0m0.000s

real 0m7.280s user 0m0.000s sys 0m0.000s

1.5.3

real 0m15.825s user 0m0.000s sys 0m0.000s

real 0m17.916s user 0m0.000s sys 0m0.000s

real 0m16.444s user 0m0.000s sys 0m0.000s

kokizzu commented 8 years ago

i got even worse result on Mac OSX 64-bit mid-2012 (i7-3xxx, 16GB, SSD)

$ rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build
go version go1.4.3 darwin/amd64

real    0m43.886s
user    0m45.049s
sys 0m2.789s
bradfitz commented 8 years ago

Much has been done on this in Go 1.7. There will be more in 1.8.

rsc commented 8 years ago

Much has been done on this in Go 1.8. There will be more in 1.9.

McGiver- commented 7 years ago

I dont understand why my friend who is compiling the same code on his i5 6200 cpu on his laptop compiles at around 1.4 seconds and my i7 6700k cpu on my desktop compiles at 6 seconds. go 1.7.3

kokizzu commented 7 years ago

maybe ssd

On Nov 6, 2016 11:36 AM, "McGyver-" notifications@github.com wrote:

I dont understand why my friend who is compiling the same code on his i5 6200 cpu on his laptop compiles at around 1.4 seconds and my i7 6700k cpu on my desktop compiles at 6 seconds. go 1.7.3

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/golang/go/issues/13646#issuecomment-258660011, or mute the thread https://github.com/notifications/unsubscribe-auth/ABAy6qBzsCBeGVbcIm7WooS2vPVsGUGMks5q7VlAgaJpZM4G3C23 .

McGiver- commented 7 years ago

@kokizzu Nope my Desktop has no hard disk only ssd. 16gb ram. Im wondering if there is something wrong with config but i dont think i have anything different especially since im the one who set up go on my friends laptop im pretty sure i performed the same steps.

McGiver- commented 7 years ago

here is my go env set GOARCH=amd64 set GOBIN=C:\Users\Something\Documents\CodeProjects\Go\bin set GOEXE=.exe set GOHOSTARCH=amd64 set GOHOSTOS=windows set GOOS=windows set GOPATH=C:\Users\Something\Documents\CodeProjects\Go set GORACE= set GOROOT=C:\Go set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64 set CC=gcc set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 set CXX=g++ set CGO_ENABLED=1

bradfitz commented 7 years ago

Let's move this conversation to a mailing list. This is just a tracking bug.

kokizzu commented 7 years ago

Performance check :3 rm -rf $GOPATH/pkg/* 2>1 > /dev/null; go version ; time go build

the exact same project as the first one, 1.5 or below not tested because the code no longer compatible.

go version go1.6.3 linux/amd64

CPU: 93.47s     Real: 36.97s    RAM: 470020KB
CPU: 91.22s     Real: 36.26s    RAM: 463844KB

go version go1.7.3 linux/amd64

CPU: 64.09s     Real: 31.05s    RAM: 353952KB
CPU: 64.95s     Real: 31.23s    RAM: 352440KB

go version go1.8beta1 linux/amd64

CPU: 56.83s     Real: 29.00s    RAM: 352572KB
CPU: 59.78s     Real: 30.26s    RAM: 354760KB
josharian commented 7 years ago

The link http://seblu.net/a/archive/packages/g/go/ leads to a 404. Do you have an updated place to go find the code which is slow to compile? Thanks.

kokizzu commented 7 years ago

@josharian seblu.net was a place to look archlinux package archive (now i'm using agetpkg-git), not code that slow to compile. The code itself is owned by my previous company i've been working to, so i can't publish it here. it's about 122K LOC (4.6 MB of source code).

current benchmark 

alias time='/usr/bin/time -f "\nCPU: %Us\tReal: %es\tRAM: %MKB"'
rm -rf $GOPATH/pkg/ ; go version ; time go build

go version go1.6.3 linux/amd64
CPU: 93.02s     Real: 37.26s    RAM: 488024KB

go version go1.7.5 linux/amd64
CPU: 66.35s     Real: 31.31s    RAM: 353620KB

go version go1.8 linux/amd64
CPU: 58.85s     Real: 29.63s    RAM: 353604KB

go 1.5 and 1.4 no longer able to compile (break with go-sqlcipher and mime/quotedprintable)

If you're looking for large go source code to compile, you can try juju or docker.

josharian commented 7 years ago

It is well known that compilation times are slower from 1.5 on and that they are getting slowly better (as your current benchmarks show).

There are many different things that may make some code particularly slow to compile. Without actual code to work with, there's nothing to be done here. I'm going to close this. If/when there's code to look at, feel free to leave a comment and I'll reopen.