search

redpanda-data / connect

Fancy stream processing made operationally mundane
https://docs.redpanda.com/redpanda-connect/about/
8.13k stars 835 forks source link

Sigfault on Mac (runtime.sigpanic) #863

Closed klintan closed 3 years ago

klintan commented 3 years ago

Hello!

Trying to run Benthos on mac and getting sigpanic

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0xb01dfacedebac1e pc=0x7fff2034ac9e]

runtime stack:
runtime: unexpected return pc for runtime.sigpanic called from 0x7fff2034ac9e
stack: frame={sp:0x7ffeefbff4e8, fp:0x7ffeefbff538} stack=[0x7ffeefb80588,0x7ffeefbff5f0)
0x00007ffeefbff3e8:  0x01007ffeefbff408  0x0000000000000004
0x00007ffeefbff3f8:  0x000000000000001f  0x00007fff2034ac9e
0x00007ffeefbff408:  0x0b01dfacedebac1e  0x0000000000000001
0x00007ffeefbff418:  0x000000000403acb1 <runtime.throw+0x0000000000000071>  0x00007ffeefbff4b8
0x00007ffeefbff428:  0x000000000615fa4b  0x00007ffeefbff470
0x00007ffeefbff438:  0x000000000403af68 <runtime.fatalthrow.func1+0x0000000000000048>  0x0000000007d4fbe0
0x00007ffeefbff448:  0x0000000000000001  0x0000000000000001
0x00007ffeefbff458:  0x00007ffeefbff4b8  0x000000000403acb1 <runtime.throw+0x0000000000000071>
0x00007ffeefbff468:  0x0000000007d4fbe0  0x00007ffeefbff4a8
0x00007ffeefbff478:  0x000000000403aef0 <runtime.fatalthrow+0x0000000000000050>  0x00007ffeefbff488
0x00007ffeefbff488:  0x000000000403af20 <runtime.fatalthrow.func1+0x0000000000000000>  0x0000000007d4fbe0
0x00007ffeefbff498:  0x000000000403acb1 <runtime.throw+0x0000000000000071>  0x00007ffeefbff4b8
0x00007ffeefbff4a8:  0x00007ffeefbff4d8  0x000000000403acb1 <runtime.throw+0x0000000000000071>
0x00007ffeefbff4b8:  0x00007ffeefbff4c0  0x000000000403ace0 <runtime.throw.func1+0x0000000000000000>
0x00007ffeefbff4c8:  0x000000000618a1bb  0x000000000000002a
0x00007ffeefbff4d8:  0x00007ffeefbff528  0x00000000040512d6 <runtime.sigpanic+0x0000000000000396>
0x00007ffeefbff4e8: <0x000000000618a1bb  0x0000000007d4fbe0
0x00007ffeefbff4f8:  0x00007ffeefbff568  0x000000000402be26 <runtime.(*mheap).allocSpan+0x0000000000000546>
0x00007ffeefbff508:  0x000000c00152c000  0x0000000000002000
0x00007ffeefbff518:  0x00007ffe00000008  0x000000000402cad1 <runtime.addspecial+0x0000000000000191>
0x00007ffeefbff528:  0x00007ffeefbff570 !0x00007fff2034ac9e
0x00007ffeefbff538: >0x00007ffeefbff570  0x0000000007be4000
0x00007ffeefbff548:  0x00000000000008bc  0x00000000047652c5 <golang.org/x/sys/unix.libc_ioctl_trampoline+0x0000000000000005>
0x00007ffeefbff558:  0x000000000407089f <runtime.syscall+0x000000000000001f>  0x000000c000cd1980
0x00007ffeefbff568:  0x000000000407020c <runtime.nanotime_trampoline+0x000000000000000c>  0x000000c000cd1950
0x00007ffeefbff578:  0x000000000406e6d0 <runtime.asmcgocall+0x0000000000000070>  0x0000000000000001
0x00007ffeefbff588:  0x0000000004011200 <runtime.notetsleep+0x0000000000000060>  0x1200000000001018
0x00007ffeefbff598:  0x0000000004031b65 <runtime.mProf_Malloc.func1+0x0000000000000025>  0x0000000007d93c18
0x00007ffeefbff5a8:  0x00000000000006d8  0x000000c0000001a0
0x00007ffeefbff5b8:  0x000000000406c7e9 <runtime.systemstack+0x0000000000000049>  0x0000000000000004
0x00007ffeefbff5c8:  0x00000000067c8600  0x0000000007d4fbe0
0x00007ffeefbff5d8:  0x00007ffeefbff628  0x000000000406c6e5 <runtime.mstart+0x0000000000000005>
0x00007ffeefbff5e8:  0x000000000406c69d <runtime.rt0_go+0x000000000000013d>
runtime.throw({0x618a1bb, 0x7d4fbe0})
    /usr/local/Cellar/go/1.17/libexec/src/runtime/panic.go:1198 +0x71
runtime: unexpected return pc for runtime.sigpanic called from 0x7fff2034ac9e
stack: frame={sp:0x7ffeefbff4e8, fp:0x7ffeefbff538} stack=[0x7ffeefb80588,0x7ffeefbff5f0)
0x00007ffeefbff3e8:  0x01007ffeefbff408  0x0000000000000004
0x00007ffeefbff3f8:  0x000000000000001f  0x00007fff2034ac9e
0x00007ffeefbff408:  0x0b01dfacedebac1e  0x0000000000000001
0x00007ffeefbff418:  0x000000000403acb1 <runtime.throw+0x0000000000000071>  0x00007ffeefbff4b8
0x00007ffeefbff428:  0x000000000615fa4b  0x00007ffeefbff470
0x00007ffeefbff438:  0x000000000403af68 <runtime.fatalthrow.func1+0x0000000000000048>  0x0000000007d4fbe0
0x00007ffeefbff448:  0x0000000000000001  0x0000000000000001
0x00007ffeefbff458:  0x00007ffeefbff4b8  0x000000000403acb1 <runtime.throw+0x0000000000000071>
0x00007ffeefbff468:  0x0000000007d4fbe0  0x00007ffeefbff4a8
0x00007ffeefbff478:  0x000000000403aef0 <runtime.fatalthrow+0x0000000000000050>  0x00007ffeefbff488
0x00007ffeefbff488:  0x000000000403af20 <runtime.fatalthrow.func1+0x0000000000000000>  0x0000000007d4fbe0
0x00007ffeefbff498:  0x000000000403acb1 <runtime.throw+0x0000000000000071>  0x00007ffeefbff4b8
0x00007ffeefbff4a8:  0x00007ffeefbff4d8  0x000000000403acb1 <runtime.throw+0x0000000000000071>
0x00007ffeefbff4b8:  0x00007ffeefbff4c0  0x000000000403ace0 <runtime.throw.func1+0x0000000000000000>
0x00007ffeefbff4c8:  0x000000000618a1bb  0x000000000000002a
0x00007ffeefbff4d8:  0x00007ffeefbff528  0x00000000040512d6 <runtime.sigpanic+0x0000000000000396>
0x00007ffeefbff4e8: <0x000000000618a1bb  0x0000000007d4fbe0
0x00007ffeefbff4f8:  0x00007ffeefbff568  0x000000000402be26 <runtime.(*mheap).allocSpan+0x0000000000000546>
0x00007ffeefbff508:  0x000000c00152c000  0x0000000000002000
0x00007ffeefbff518:  0x00007ffe00000008  0x000000000402cad1 <runtime.addspecial+0x0000000000000191>
0x00007ffeefbff528:  0x00007ffeefbff570 !0x00007fff2034ac9e
0x00007ffeefbff538: >0x00007ffeefbff570  0x0000000007be4000
0x00007ffeefbff548:  0x00000000000008bc  0x00000000047652c5 <golang.org/x/sys/unix.libc_ioctl_trampoline+0x0000000000000005>
0x00007ffeefbff558:  0x000000000407089f <runtime.syscall+0x000000000000001f>  0x000000c000cd1980
0x00007ffeefbff568:  0x000000000407020c <runtime.nanotime_trampoline+0x000000000000000c>  0x000000c000cd1950
0x00007ffeefbff578:  0x000000000406e6d0 <runtime.asmcgocall+0x0000000000000070>  0x0000000000000001
0x00007ffeefbff588:  0x0000000004011200 <runtime.notetsleep+0x0000000000000060>  0x1200000000001018
0x00007ffeefbff598:  0x0000000004031b65 <runtime.mProf_Malloc.func1+0x0000000000000025>  0x0000000007d93c18
0x00007ffeefbff5a8:  0x00000000000006d8  0x000000c0000001a0
0x00007ffeefbff5b8:  0x000000000406c7e9 <runtime.systemstack+0x0000000000000049>  0x0000000000000004
0x00007ffeefbff5c8:  0x00000000067c8600  0x0000000007d4fbe0
0x00007ffeefbff5d8:  0x00007ffeefbff628  0x000000000406c6e5 <runtime.mstart+0x0000000000000005>
0x00007ffeefbff5e8:  0x000000000406c69d <runtime.rt0_go+0x000000000000013d>
runtime.sigpanic()
    /usr/local/Cellar/go/1.17/libexec/src/runtime/signal_unix.go:719 +0x396

goroutine 1 [syscall, locked to thread]:
syscall.syscall(0x47652c0, 0x1, 0x40487413, 0xc000cd1a10)
    /usr/local/Cellar/go/1.17/libexec/src/runtime/sys_darwin.go:22 +0x3b fp=0xc000cd1980 sp=0xc000cd1960 pc=0x406b0db
syscall.syscall(0x40ba5e6, 0x60, 0xc000cd1a38, 0x40ba518)
    <autogenerated>:1 +0x26 fp=0xc000cd19c8 sp=0xc000cd1980 pc=0x40710a6
golang.org/x/sys/unix.ioctl(0x61194ff, 0x4, 0x1000000000001)
    /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/golang.org/x/sys@v0.0.0-20210423082822-04245dca01da/unix/zsyscall_darwin_amd64.go:690 +0x39 fp=0xc000cd19f8 sp=0xc000cd19c8 pc=0x4764f59
golang.org/x/sys/unix.IoctlGetTermios(...)
    /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/golang.org/x/sys@v0.0.0-20210423082822-04245dca01da/unix/ioctl.go:73
github.com/mattn/go-isatty.IsTerminal(0x61194ff)
    /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/mattn/go-isatty@v0.0.12/isatty_bsd.go:10 +0x50 fp=0xc000cd1a68 sp=0xc000cd19f8 pc=0x57f7b10
github.com/fatih/color.init()
    /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/fatih/color@v1.10.0/color.go:21 +0x7a fp=0xc000cd1aa0 sp=0xc000cd1a68 pc=0x57f7f7a
runtime.doInit(0x7bf3240)
    /usr/local/Cellar/go/1.17/libexec/src/runtime/proc.go:6498 +0x123 fp=0xc000cd1bd8 sp=0xc000cd1aa0 pc=0x404a5c3
runtime.doInit(0x7c023e0)
    /usr/local/Cellar/go/1.17/libexec/src/runtime/proc.go:6475 +0x71 fp=0xc000cd1d10 sp=0xc000cd1bd8 pc=0x404a511
runtime.doInit(0x7c08840)
    /usr/local/Cellar/go/1.17/libexec/src/runtime/proc.go:6475 +0x71 fp=0xc000cd1e48 sp=0xc000cd1d10 pc=0x404a511
runtime.doInit(0x7be8f00)
    /usr/local/Cellar/go/1.17/libexec/src/runtime/proc.go:6475 +0x71 fp=0xc000cd1f80 sp=0xc000cd1e48 pc=0x404a511
runtime.main()
    /usr/local/Cellar/go/1.17/libexec/src/runtime/proc.go:238 +0x1e6 fp=0xc000cd1fe0 sp=0xc000cd1f80 pc=0x403d306
runtime.goexit()
    /usr/local/Cellar/go/1.17/libexec/src/runtime/asm_amd64.s:1581 +0x1 fp=0xc000cd1fe8 sp=0xc000cd1fe0 pc=0x406e9c1

goroutine 6 [chan receive]:
github.com/ClickHouse/clickhouse-go.init.0.func1()
    /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/!click!house/clickhouse-go@v1.4.3/bootstrap.go:50 +0x2d
created by github.com/ClickHouse/clickhouse-go.init.0
    /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/github.com/!click!house/clickhouse-go@v1.4.3/bootstrap.go:47 +0x45

goroutine 50 [select]:
go.opencensus.io/stats/view.(*worker).start(0xc000320b80)
    /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/go.opencensus.io@v0.22.5/stats/view/worker.go:276 +0xb9
created by go.opencensus.io/stats/view.init.0
    /Users/brew/Library/Caches/Homebrew/go_mod_cache/pkg/mod/go.opencensus.io@v0.22.5/stats/view/worker.go:34 +0x92

Intel Mac MacBook Pro (15-inch, 2019) Big sur 11.5.2 .

Ran brew update and brew upgrade and tried reinstall Benthosbrew reinstall benthos

Any ideas what it might be ?

Jeffail commented 3 years ago

Hey @klintan, uh oh, that looks like an exceptional sad. I suspect this is due to homebrew compiling with Go 1.17, theres similar panics out there where a few dependencies needed upgrading so hopefully it'll that simple.

Thanks for raising this, I can likely get a patch release out this weekend but in the meantime you'll lkely need to downgrade if possible.

Jeffail commented 3 years ago

I'm not entirely sure which dependency update fixed it but it looks as though this is fixed by the latest commit, so I'll port the dependencies over and cut a patch release.

Jeffail commented 3 years ago

Looks like this is fixed with, tagging shortly: https://github.com/Jeffail/benthos/commit/de00ee98b7314545cae84f58c4e792eebc627fbf

Jeffail commented 3 years ago

Released: https://github.com/Jeffail/benthos/releases/tag/v3.54.1, it'll take a little while for homebrew to update but you can pull the latest binary from the releases page or use the slightly dodge curl installer curl -Lsf https://sh.benthos.dev | bash. If you still find issues in the newer version please reopen and I'll dig a bit deeper.

klintan commented 3 years ago

Awesome thank you so much Ash! :)