search

ElementsProject / lightning

Core Lightning — Lightning Network implementation focusing on spec compliance and performance
Other
2.84k stars 901 forks source link

renepay crashes #7126

Closed Sjors closed 7 months ago

Sjors commented 8 months ago

Issue and Steps to Reproduce

Running v24.02

lightning-cli renepay lnbc1...
{
   "code": 207,
   "message": "Invoice expired"
}

Get fresh invoice...

lightning-cli renepay lnbc1...
{
   "code": -4,
   "message": "Plugin terminated before replying to RPC call."
}
{
   "code": -32601,
   "message": "Unknown command 'renepay'"
}

In the log, not sure at which step this happened though:

2024-03-04T09:28:55.061Z DEBUG   plugin-cln-renepay: Starting renepay
2024-03-04T09:28:55.555Z INFO    plugin-cln-renepay: Killing plugin: exited during normal operation
Sjors commented 8 months ago

Is there a convenient way to restart renepay without restarting the whole node?

vincenzopalazzo commented 8 months ago

Is there a convenient way to restart renepay without restarting the whole node?

idk if lightning-cli plugin start renepay works, but it should work so maybe we should open a feature request.

Other than that on my system I ran 

➜  VincentSSD cat .lightning/testnet/log.log | grep "renepay"
2024-02-13T14:24:55.770Z DEBUG   plugin-manager: started(211099) /usr/local/libexec/c-lightning/plugins/cln-renepay
2024-02-13T14:25:18.264Z DEBUG   plugin-manager: started(211204) /usr/local/libexec/c-lightning/plugins/cln-renepay

and then I can do lightning-cli plugin start /usr/local/libexec/c-lightning/plugins/cln-renepay

Hope that this will help

vincenzopalazzo commented 8 months ago

Ah, do you have the stacktrace of the crash? it is missing inside the PR description but it should be inside the logs e.g: https://github.com/ElementsProject/lightning/issues/7115

Sjors commented 8 months ago 
~$ lightning-cli plugin start renepay
{
   "code": -32602,
   "message": "/home/lightning/.lightning/plugins/renepay is not executable: No such file or directory"
}

There's no stack trace in the log, which is set to log-level=debug.

Found the executable, it's in /home/lightning/libexec/c-lightning/plugins/cln-renepay for me. But:

$ lightning-cli plugin start /home/lightning/libexec/c-lightning/plugins/cln-renepay
{
   "code": -32602,
   "message": "/home/lightning/libexec/c-lightning/plugins/cln-renepay: Invalid argument"
}

(but I think I restarted the node since the last crash, so maybe this is a cryptic way of saying that the plugin is already running)

Sjors commented 8 months ago

Oh actually, although there's nothing the in the log file, there is something in the systemd journal:

Mar 04 09:28:55 sjimmie lightningd[1049218]: cln-renepay: plugins/renepay/mcf.c:1233: pseudorand_interval: Assertion `b > a' failed.
Mar 04 09:28:55 sjimmie lightningd[1049218]: cln-renepay: FATAL SIGNAL 6 (version v24.02)
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb751d15d send_backtrace
Mar 04 09:28:55 sjimmie lightningd[1049218]:         common/daemon.c:33
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb751d1f4 crashdump
Mar 04 09:28:55 sjimmie lightningd[1049218]:         common/daemon.c:75
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x7fad6e44298f ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x7fad6e499a1b ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x7fad6e4428e5 ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x7fad6e4268b6 ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x7fad6e4267da ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x7fad6e439205 ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb7506669 pseudorand_interval
Mar 04 09:28:55 sjimmie lightningd[1049218]:         plugins/renepay/mcf.c:1233
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb7506afb get_flow_paths
Mar 04 09:28:55 sjimmie lightningd[1049218]:         plugins/renepay/mcf.c:1378
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb75075a2 minflow
Mar 04 09:28:55 sjimmie lightningd[1049218]:         plugins/renepay/mcf.c:1714
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb75011c9 add_payflows
Mar 04 09:28:55 sjimmie lightningd[1049218]:         plugins/renepay/pay_flow.c:481
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb74ff9eb try_paying
Mar 04 09:28:55 sjimmie lightningd[1049218]:         plugins/renepay/pay.c:433
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb74ffca0 listpeerchannels_done
Mar 04 09:28:55 sjimmie lightningd[1049218]:         plugins/renepay/pay.c:511
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb750e281 handle_rpc_reply
Mar 04 09:28:55 sjimmie lightningd[1049218]:         plugins/libplugin.c:961
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb750e424 rpc_read_response_one
Mar 04 09:28:55 sjimmie lightningd[1049218]:         plugins/libplugin.c:1148
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb750e4d5 rpc_conn_read_response
Mar 04 09:28:55 sjimmie lightningd[1049218]:         plugins/libplugin.c:1172
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb753e7b2 next_plan
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ccan/ccan/io/io.c:59
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb753ec7f do_plan
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ccan/ccan/io/io.c:407
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb753ed1c io_ready
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ccan/ccan/io/io.c:417
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb7540628 io_loop
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ccan/ccan/io/poll.c:453
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb750e870 plugin_main
Mar 04 09:28:55 sjimmie lightningd[1049218]:         plugins/libplugin.c:2086
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb74ffdcd main
Mar 04 09:28:55 sjimmie lightningd[1049218]:         plugins/renepay/pay.c:1424
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x7fad6e42814f ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x7fad6e428208 ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb74fc974 ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0xffffffffffffffff ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0
Mar 04 09:28:55 sjimmie lightningd[1049218]: cln-renepay: FATAL SIGNAL 11 (version v24.02)
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb751d15d send_backtrace
Mar 04 09:28:55 sjimmie lightningd[1049218]:         common/daemon.c:33
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x560bb751d1f4 crashdump
Mar 04 09:28:55 sjimmie lightningd[1049218]:         common/daemon.c:75
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x7fad6e44298f ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0
Mar 04 09:28:55 sjimmie lightningd[1049218]: 0x0 ???
Mar 04 09:28:55 sjimmie lightningd[1049218]:         ???:0

Seems to be the same as #7115, but this was a regular bolt11 invoice.

vincenzopalazzo commented 8 months ago

oh nice thanks, maybe we have different report location because I do not use journal 😄

Thanks for the stacktrace

cdecker commented 7 months ago

Closing as a duplicate of #7115