search

ElementsProject / lightning

Core Lightning — Lightning Network implementation focusing on spec compliance and performance
Other
2.84k stars 901 forks source link

crash in master on ppc64le #7039

Closed grubles closed 8 months ago

grubles commented 9 months ago

OS: Fedora 39 Arch: ppc64le CLN at 25b40d2ce89c91993f0b1746505f46d090092aab

Shortly after starting lightningd it crashes: 

2024-02-04T17:52:58.872Z **BROKEN** lightningd: FATAL SIGNAL 11 (version v23.11-308-g25b40d2-modded)
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0x100c511b
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: common/daemon.c:75 (crashdump) 0x100c5193
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7fff8c480463
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: bitcoin/short_channel_id.c:96 (towire_short_channel_id) 0x100f0874
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: channeld/channeld_wiregen.c:287 (towire_channeld_init) 0x1011f2c3
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: lightningd/channel_control.c:1588 (peer_start_channeld) 0x10021837
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: lightningd/peer_control.c:1309 (connect_activate_subd) 0x1006ac07
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: lightningd/peer_control.c:1408 (peer_connected_hook_final) 0x1006e8bb
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:194 (plugin_hook_call_next) 0x10085a27
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:169 (plugin_hook_callback) 0x10085c37
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:661 (plugin_response_handle) 0x1007cd9b
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:773 (plugin_read_json_one) 0x1008279b
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:824 (plugin_read_json) 0x10082ae3
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x101f5e47
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x101f6723
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x101f682b
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0x101f92ff
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x10043c6f
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1420 (main) 0x1004c193
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7fff8bc30f2b
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x7fff8bc3116b
2024-02-04T17:52:58.872Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff
grubles commented 9 months ago

0575f8a544455519d3a70db49c61eda0a0fbc2a1 is the oldest commit I can run without CLN refusing to downgrade its db. Unfortunately the crash still occurs.

grubles commented 9 months ago

This happens right before the crash every time:

2024-02-05T02:21:08.508Z DEBUG   REDACTED-connectd: peer_out WIRE_INIT
2024-02-05T02:21:08.508Z DEBUG   gossipd: REPLY WIRE_GOSSIPD_GET_ADDRS_REPLY with 0 fds
2024-02-05T02:21:08.508Z DEBUG   REDACTED-connectd: peer_in WIRE_INIT
2024-02-05T02:21:08.508Z DEBUG   lightningd: Calling peer_connected hook of plugin chanbackup
2024-02-05T02:21:08.508Z DEBUG   REDACTED-connectd: Handed peer, entering loop
2024-02-05T02:21:08.509Z DEBUG   lightningd: Plugin chanbackup returned from peer_connected hook call
2024-02-05T02:21:08.509Z DEBUG   REDACTED-chan#9447: Peer has reconnected, state CHANNELD_NORMAL: connecting subd
2024-02-05T02:21:08.513Z DEBUG   REDACTED-channeld-chan#9447: pid 200187, msgfd 41
2024-02-05T02:21:08.513Z DEBUG   REDACTED-chan#9447: Already have funding locked in
rustyrussell commented 8 months ago

OK, this looks very much like an old channel without an alias. We should provide one....

vincenzopalazzo commented 8 months ago

This happens also to me yesterday while I was doing bug fixing on folgore, and I was on a basic x86 machine

ORMAL: connecting subd
2024-02-06T20:55:25.243Z DEBUG   02312627fdf07fbdd7e5ddb136611bdde9b00d26821d14d94891395452f67af248-channeld-chan#77: pid 148407, msgfd 83
2024-02-06T20:55:25.245Z DEBUG   02312627fdf07fbdd7e5ddb136611bdde9b00d26821d14d94891395452f67af248-chan#77: Already have funding locked in
2024-02-06T20:55:25.246Z UNUSUAL 02312627fdf07fbdd7e5ddb136611bdde9b00d26821d14d94891395452f67af248-chan#77: Ignoring fee limits!
2024-02-06T20:55:27.153Z **BROKEN** lightningd: FATAL SIGNAL 11 (version v23.11-312-g4fc9e71)
2024-02-06T20:55:27.155Z **BROKEN** lightningd: backtrace: common/daemon.c:38 (send_backtrace) 0x19f379
2024-02-06T20:55:27.171Z **BROKEN** lightningd: backtrace: common/daemon.c:75 (crashdump) 0x19f3b9
2024-02-06T20:55:27.172Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x4b4776f
2024-02-06T20:55:27.173Z **BROKEN** lightningd: backtrace: bitcoin/short_channel_id.c:96 (towire_short_channel_id) 0x1bac19
2024-02-06T20:55:27.174Z **BROKEN** lightningd: backtrace: channeld/channeld_wiregen.c:287 (towire_channeld_init) 0x1d9b22
2024-02-06T20:55:27.176Z **BROKEN** lightningd: backtrace: lightningd/channel_control.c:1588 (peer_start_channeld) 0x12de3c
2024-02-06T20:55:27.177Z **BROKEN** lightningd: backtrace: lightningd/peer_control.c:1309 (connect_activate_subd) 0x161d76
2024-02-06T20:55:27.178Z **BROKEN** lightningd: backtrace: lightningd/peer_control.c:1408 (peer_connected_hook_final) 0x164a2d
2024-02-06T20:55:27.179Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:194 (plugin_hook_call_next) 0x1746ea
2024-02-06T20:55:27.180Z **BROKEN** lightningd: backtrace: lightningd/plugin_hook.c:169 (plugin_hook_callback) 0x1748aa
2024-02-06T20:55:27.181Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:662 (plugin_response_handle) 0x16ef6f
2024-02-06T20:55:27.182Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:774 (plugin_read_json_one) 0x17274d
2024-02-06T20:55:27.183Z **BROKEN** lightningd: backtrace: lightningd/plugin.c:825 (plugin_read_json) 0x1729e6
2024-02-06T20:55:27.184Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:59 (next_plan) 0x30929a
2024-02-06T20:55:27.185Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:407 (do_plan) 0x309721
2024-02-06T20:55:27.186Z **BROKEN** lightningd: backtrace: ccan/ccan/io/io.c:417 (io_ready) 0x3097ba
2024-02-06T20:55:27.187Z **BROKEN** lightningd: backtrace: ccan/ccan/io/poll.c:453 (io_loop) 0x30b01a
2024-02-06T20:55:27.188Z **BROKEN** lightningd: backtrace: lightningd/io_loop_with_timers.c:22 (io_loop_with_timers) 0x146d27
2024-02-06T20:55:27.189Z **BROKEN** lightningd: backtrace: lightningd/lightningd.c:1420 (main) 0x14c270
2024-02-06T20:55:27.190Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x4b30ccf
2024-02-06T20:55:27.191Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x4b30d89
2024-02-06T20:55:27.192Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0x121ce4
2024-02-06T20:55:27.193Z **BROKEN** lightningd: backtrace: (null):0 ((null)) 0xffffffffffffffff