golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
124.27k stars 17.7k forks source link

runtime: windows-amd64-race builder fails with errno=1455 #22553

Open alexbrainman opened 7 years ago

alexbrainman commented 7 years ago

I noticed some failures on windows-amd64-race builder:

https://build.golang.org/log/51b118b069de539851ff1cc07769de863bf47797

--- FAIL: TestRace (2.21s)
    race_test.go:71: failed to parse test output:
        # command-line-arguments_test
        runtime: VirtualAlloc of 1048576 bytes failed with errno=1455
        fatal error: out of memory
...
FAIL
FAIL    runtime/race    40.936s

and

https://build.golang.org/log/f73793faa51acd4b45e9ef88a18e462a4913448c

==2820==ERROR: ThreadSanitizer failed to allocate 0x000000400000 (4194304) bytes at 0x040177c00000 (error code: 1455)
runtime: newstack sp=0x529fdf0 stack=[0xc042200000, 0xc042202000]
    morebuf={pc:0x4083a4 sp:0x529fe00 lr:0x0}
    sched={pc:0x47bec7 sp:0x529fdf8 lr:0x0 ctxt:0x0}
runtime: gp=0xc0421ec300, gp->status=0x2
 runtime: split stack overflow: 0x529fdf0 < 0xc042200000
fatal error: runtime: split stack overflow
...
FAIL    runtime/trace   14.859s
...
--- FAIL: TestMutexMisuse (0.28s)
    mutex_test.go:173: Mutex.Unlock: did not find failure with message about unlocked lock: fork/exec C:\Users\gopher\AppData\Local\Temp\go-build022577002\b075\sync.test.exe: The paging file is too small for this operation to complete.

    mutex_test.go:173: Mutex.Unlock2: did not find failure with message about unlocked lock: fork/exec C:\Users\gopher\AppData\Local\Temp\go-build022577002\b075\sync.test.exe: The paging file is too small for this operation to complete.

    mutex_test.go:173: RWMutex.Unlock: did not find failure with message about unlocked lock: fork/exec C:\Users\gopher\AppData\Local\Temp\go-build022577002\b075\sync.test.exe: The paging file is too small for this operation to complete.

    mutex_test.go:173: RWMutex.Unlock2: did not find failure with message about unlocked lock: fork/exec C:\Users\gopher\AppData\Local\Temp\go-build022577002\b075\sync.test.exe: The paging file is too small for this operation to complete.

    mutex_test.go:173: RWMutex.Unlock3: did not find failure with message about unlocked lock: fork/exec C:\Users\gopher\AppData\Local\Temp\go-build022577002\b075\sync.test.exe: The paging file is too small for this operation to complete.

    mutex_test.go:173: RWMutex.RUnlock: did not find failure with message about unlocked lock: fork/exec C:\Users\gopher\AppData\Local\Temp\go-build022577002\b075\sync.test.exe: The paging file is too small for this operation to complete.

    mutex_test.go:173: RWMutex.RUnlock2: did not find failure with message about unlocked lock: fork/exec C:\Users\gopher\AppData\Local\Temp\go-build022577002\b075\sync.test.exe: The paging file is too small for this operation to complete.

    mutex_test.go:173: RWMutex.RUnlock3: did not find failure with message about unlocked lock: fork/exec C:\Users\gopher\AppData\Local\Temp\go-build022577002\b075\sync.test.exe: The paging file is too small for this operation to complete.

FAIL
FAIL    sync    2.803s

1455 is ERROR_COMMITMENT_LIMIT "The paging file is too small for this operation to complete."

Perhaps we just need to make builders page file larger or give PC more memory or something.

Alex

bradfitz commented 7 years ago

The Windows builders are GCE VMs of size n1-highcpu-4, which is 3.6 GB of RAM.

I'm more interested in knowing why Go now needs more than ~3 GB of RAM to compile.

alexbrainman commented 7 years ago

I'm more interested in knowing why Go now needs more than ~3 GB of RAM to compile.

I do not know. Note that this is only -race builder. Also I just submitted CL 74490 that fixed the problem where each thread used 2MB instead of 2KB of memory. But I don't see how this CL might help in scenarios I reported.

Alex

aclements commented 7 years ago

This has been happening occasionally since August:

$ greplogs -dashboard -E errno=1455 -l -md 2016-01-28T22:52:42-2f57d0f/windows-amd64-gce (some other bug) 2017-08-16T19:04:00-8127dbf/windows-amd64-race 2017-08-28T22:46:30-93471a1/windows-amd64-race 2017-09-18T17:19:43-6c8d512/windows-amd64-race 2017-10-05T18:17:27-69b0b3a/windows-amd64-race 2017-10-31T22:36:43-6fac139/windows-amd64-race 2017-11-01T18:23:58-7bd0b61/windows-amd64-race 2017-11-02T03:32:31-1114d40/windows-amd64-race 2017-11-06T21:32:06-0c55495/windows-amd64-race

benitogf commented 3 years ago

hello this is showing on the github actions hosted runner: https://github.com/benitogf/level/pull/3/checks?check_run_id=2780549553#step:7:32

I tried using https://github.com/marketplace/actions/configure-pagefile to increase the pagefile but it doesn't solve the issue