petrkotek
commented
3 months ago +1.
I've run into this issue when trying to use geojson.Feature (https://github.com/paulmach/go.geojson/blob/master/feature.go) in the pipeline.
This doesn't work as the geojson.Geometry is recursive.
As a workaround, as soon as I decode the geojson.Feature from my input, I convert it into an internal struct which doesn't have the recursion (luckily I don't really need to recursion) -- and all functions in the pipeline use this internal struct.
Stack trace (for discoverability):
runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0xc03f4003b8 stack=[0xc03f400000, 0xc05f400000]
fatal error: stack overflow
runtime stack:
runtime.throw({0x6f2cfc4?, 0x70000abcddd8?})
/usr/local/Cellar/go/1.22.0/libexec/src/runtime/panic.go:1023 +0x5c fp=0x70000abcdd88 sp=0x70000abcdd58 pc=0x5ce609c
runtime.newstack()
/usr/local/Cellar/go/1.22.0/libexec/src/runtime/stack.go:1103 +0x5bd fp=0x70000abcdf38 sp=0x70000abcdd88 pc=0x5d01fdd
runtime.morestack()
/usr/local/Cellar/go/1.22.0/libexec/src/runtime/asm_amd64.s:616 +0x7a fp=0x70000abcdf40 sp=0x70000abcdf38 pc=0x5d1f0ba
goroutine 1 gp=0xc0000061c0 m=10 mp=0xc000700008 [running]:
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/graphx/schema.(*Registry).logicalTypeToFieldType(0x9297c20, {0x7a7e148, 0x7241420})
/Users/petrkotek/go/pkg/mod/github.com/apache/beam/sdks/v2@v2.56.0/go/pkg/beam/core/runtime/graphx/schema/schema.go:330 +0x4fa fp=0xc03f4003c8 sp=0xc03f4003c0 pc=0x621329a
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/graphx/schema.(*Registry).reflectTypeToFieldType(0x9297c20, {0x7a7e148, 0x7241420})
/Users/petrkotek/go/pkg/mod/github.com/apache/beam/sdks/v2@v2.56.0/go/pkg/beam/core/runtime/graphx/schema/schema.go:569 +0x37 fp=0xc03f4004a8 sp=0xc03f4003c8 pc=0x6214c97
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/graphx/schema.(*Registry).logicalTypeToFieldType(0x9297c20, {0x7a7e148, 0x72412e0})
/Users/petrkotek/go/pkg/mod/github.com/apache/beam/sdks/v2@v2.56.0/go/pkg/beam/core/runtime/graphx/schema/schema.go:335 +0xd9 fp=0xc03f400620 sp=0xc03f4004a8 pc=0x6212e79
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/graphx/schema.(*Registry).reflectTypeToFieldType(0x9297c20, {0x7a7e148, 0x72412e0})
/Users/petrkotek/go/pkg/mod/github.com/apache/beam/sdks/v2@v2.56.0/go/pkg/beam/core/runtime/graphx/schema/schema.go:569 +0x37 fp=0xc03f400700 sp=0xc03f400620 pc=0x6214c97
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/graphx/schema.(*Registry).structFieldToField(0x9297c20, {{0x710f910, 0x4}, {0x0, 0x0}, {0x7a7e148, 0x72412e0}, {0x710f915, 0xb}, 0x0, ...})
/Users/petrkotek/go/pkg/mod/github.com/apache/beam/sdks/v2@v2.56.0/go/pkg/beam/core/runtime/graphx/schema/schema.go:558 +0x8b fp=0xc03f400748 sp=0xc03f400700 pc=0x6214bcb
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/graphx/schema.(*Registry).structToSchema(0x9297c20, {0x7a7e148, 0x77c6660})
/Users/petrkotek/go/pkg/mod/github.com/apache/beam/sdks/v2@v2.56.0/go/pkg/beam/core/runtime/graphx/schema/schema.go:533 +0x627 fp=0xc03f400968 sp=0xc03f400748 pc=0x62145c7
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/graphx/schema.(*Registry).reflectTypeToFieldType(0x9297c20, {0x7a7e148, 0x77c6660})
/Users/petrkotek/go/pkg/mod/github.com/apache/beam/sdks/v2@v2.56.0/go/pkg/beam/core/runtime/graphx/schema/schema.go:612 +0x52b fp=0xc03f400a48 sp=0xc03f400968 pc=0x621518b
github.com/apache/beam/sdks/v2/go/pkg/beam/core/runtime/graphx/schema.(*Registry).reflectTypeToFieldType(0x9297c20, {0x7a7e148, 0x76a8a60})
(... and much more ...)
We do not allow types such as:
Use of these will require type registration.
Imported from Jira BEAM-3864. Original Jira may contain additional context. Reported by: herohde.