cmd/compile: autotemps can make stack frame too large

[randall77]

When the order pass introduces temporaries, it always allocates them on the stack, even if they are too big for the stack.

package main

import "reflect"

func f() {
    g(reflect.TypeOf([2000000000]*byte{}))
}

//go:noescape
func g(p reflect.Type)

We should use the same rules as escape analysis does to decide if we should put the temporary on the stack or the heap.

Before order:

.   .   .   AS2-list
.   .   .   .   NAME-reflect.i a(true) l(1376) x(0) class(PAUTO) tc(1) addrtaken assigned used INTER-interface {}
.   .   .   AS2-rlist
.   .   .   .   CONVIFACE l(6) esc(no) tc(1) implicit(true) INTER-interface {}
.   .   .   .   .   ARRAYLIT l(6) esc(h) tc(1) ARRAY-[2000000000]*byte

After order:

.   AS l(6) tc(1)
.   .   NAME-main..autotmp_5 a(true) l(6) x(0) class(PAUTO) esc(N) tc(1) assigned used ARRAY-[2000000000]*byte
.   .   ARRAYLIT l(6) esc(h) tc(1) ARRAY-[2000000000]*byte

.   AS2 l(6) tc(1)
.   AS2-list
.   .   NAME-reflect.i a(true) l(1376) x(0) class(PAUTO) tc(1) addrtaken assigned used INTER-interface {}
.   AS2-rlist
.   .   CONVIFACE l(6) esc(no) tc(1) implicit(true) INTER-interface {}
.   .   .   NAME-main..autotmp_5 a(true) l(6) x(0) class(PAUTO) esc(N) tc(1) assigned used ARRAY-[2000000000]*byte

.   VARKILL l(6) tc(1)
.   .   NAME-main..autotmp_5 a(true) l(6) x(0) class(PAUTO) esc(N) tc(1) assigned used ARRAY-[2000000000]*byte

[randall77]

This is the error:

$ go tool compile tmp2.go
tmp2.go:5:6: stack frame too large (>1GB)

[josharian]

The cutoff (1<<16) is also duplicated in a couple of places in walk.go (func isSmallMakeSlice, case ONEW in func walkexpr). Should unify all of them.

[cuonglm]

@randall77 @josharian @mdempsky How can we make autotemps heap alloc this case? The order happens after escape analysis was done.

[randall77]

@cuonglm This can be tricky. On first glance, there's not immediate problem - order can introduce calls to newobject. If the object is pointerless, it works fine. But if the object has pointers, then things that previously didn't escape now do. I think the right, athough difficult, solution would be to treat these heap-allocated things as stack somehow (no write barriers, scanned as part of stack scanning, etc.). A good first step would be to solve this issue just for pointerless objects.

[zigo101]

It looks this is also true for slice literals.

package main

import t "testing"

const M = 1024 * 1024 * 100

var n int = M - 1
var b byte

func f() {
    type _ int
    var bs = []byte{M+M: 0} 
    b = bs[n]
}

func main() {
    x := t.AllocsPerRun(2, f)
    println(int(x)) // 0
}

{edit] Some array literal corner cases:

package main

const N = 100 * 1024 * 1024

var m = make(map[byte][N]byte, 1)
func foo() {
    m[0] = [N]byte{} // copy stack to heap
}

var i interface{}
func bar() {
    i = [N]byte{} // copy stack to heap
}

func pen(vx ...interface{}) {
    type _ int
}

func qin(m map[byte][N]byte) {
    m[1] = [N]byte{} // copy stack to heap
}

func main() {
    run := func(f func(), c chan struct{}) {
        defer close(c)
        var x int
        println(&x) // <address 1>
        f()
        println(&x) // <address 2>
    }
    c1 := make(chan struct{})
    go run(foo, c1)
    <-c1
    c2 := make(chan struct{})
    go run(bar, c2)
    <-c2
    c3 := make(chan struct{})
    go run(func() {qin(map[byte][N]byte{})}, c3)
    <-c3
    c4 := make(chan struct{})
    go run(func() {pen([N]byte{})}, c4)
    <-c4
}