Crash on startup when --developer used

[ksedgwic]

version: lightningd: v23.11-14-g2f5f098-modded (VLS build on v23.11)

relevant config:

log-level=info
clboss-auto-close=true
important-plugin=/usr/local/bin/clboss
developer

result:

Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:24.681Z INFO    lightningd: v23.11-14-g2f5f098-modded
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.278Z DEBUG   lightningd: Opened log file -
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.278Z DEBUG   lightningd: Opened log file /home/cln/.lightning/testnet/lightning.log
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.305Z DEBUG   plugin-manager: started(669961) /usr/local/libexec/c-lightning/plugins/autoclean
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.345Z DEBUG   plugin-manager: started(669962) /usr/local/libexec/c-lightning/plugins/chanbackup
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.370Z DEBUG   plugin-manager: started(669963) /usr/local/libexec/c-lightning/plugins/bcli
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.395Z DEBUG   plugin-manager: started(669964) /usr/local/libexec/c-lightning/plugins/commando
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.422Z DEBUG   plugin-manager: started(669968) /usr/local/libexec/c-lightning/plugins/fetchinvoice
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.447Z DEBUG   plugin-manager: started(669969) /usr/local/libexec/c-lightning/plugins/funder
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.471Z DEBUG   plugin-manager: started(669970) /usr/local/libexec/c-lightning/plugins/topology
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.496Z DEBUG   plugin-manager: started(669971) /usr/local/libexec/c-lightning/plugins/keysend
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.519Z DEBUG   plugin-manager: started(669972) /usr/local/libexec/c-lightning/plugins/offers
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.551Z DEBUG   plugin-manager: started(669973) /usr/local/libexec/c-lightning/plugins/pay
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.589Z DEBUG   plugin-manager: started(669974) /usr/local/libexec/c-lightning/plugins/txprepare
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.619Z DEBUG   plugin-manager: started(669975) /usr/local/libexec/c-lightning/plugins/cln-renepay
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.647Z DEBUG   plugin-manager: started(669976) /usr/local/libexec/c-lightning/plugins/spenderp
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.686Z DEBUG   plugin-manager: started(669977) /usr/local/libexec/c-lightning/plugins/sql
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.713Z DEBUG   plugin-manager: started(669978) /usr/local/libexec/c-lightning/plugins/cln-grpc
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.738Z DEBUG   plugin-manager: started(669980) /usr/local/libexec/c-lightning/plugins/bookkeeper
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.762Z DEBUG   plugin-manager: started(669981) /usr/local/libexec/c-lightning/plugins/clnrest/clnrest.py
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.813Z DEBUG   plugin-manager: started(669982) /usr/local/bin/clboss
Dec 06 13:38:25 home4 vlsd2[669957]: [2023-12-06 21:38:25.853 vlsd2/vls_proxy::grpc::signer WARN] error connecting to node, will retry: transport error - error trying to connect: tcp connect error: Connection refused (os error 111)
Dec 06 13:38:25 home4 lightningd[669956]: 2023-12-06T21:38:25.886Z DEBUG   plugin-manager: started(669983) /usr/local/src/plugins/summary/summary.py
Dec 06 13:38:25 home4 lightningd[669956]: lightningd: FATAL SIGNAL 11 (version v23.11-14-g2f5f098-modded)
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c802e1f0a send_backtrace
Dec 06 13:38:26 home4 lightningd[669956]:         common/daemon.c:33
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c802e20f5 crashdump
Dec 06 13:38:26 home4 lightningd[669956]:         common/daemon.c:75
Dec 06 13:38:26 home4 lightningd[669956]: 0x7f379504251f ???
Dec 06 13:38:26 home4 lightningd[669956]:         ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c8029d1c5 plugin_conn_finish
Dec 06 13:38:26 home4 lightningd[669956]:         lightningd/plugin.c:875
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c80475970 destroy_conn
Dec 06 13:38:26 home4 lightningd[669956]:         ccan/ccan/io/poll.c:246
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c80475994 destroy_conn_close_fd
Dec 06 13:38:26 home4 lightningd[669956]:         ccan/ccan/io/poll.c:252
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c8048548f notify
Dec 06 13:38:26 home4 lightningd[669956]:         ccan/ccan/tal/tal.c:246
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c80485a70 del_tree
Dec 06 13:38:26 home4 lightningd[669956]:         ccan/ccan/tal/tal.c:437
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c80485e16 tal_free
Dec 06 13:38:26 home4 lightningd[669956]:         ccan/ccan/tal/tal.c:521
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c80473ebb io_close
Dec 06 13:38:26 home4 lightningd[669956]:         ccan/ccan/io/io.c:450
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c804760fa io_loop
Dec 06 13:38:26 home4 lightningd[669956]:         ccan/ccan/io/poll.c:457
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c80260da5 io_loop_with_timers
Dec 06 13:38:26 home4 lightningd[669956]:         lightningd/io_loop_with_timers.c:22
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c802a0986 plugins_init
Dec 06 13:38:26 home4 lightningd[669956]:         lightningd/plugin.c:1991
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c80267fd6 main
Dec 06 13:38:26 home4 lightningd[669956]:         lightningd/lightningd.c:1082
Dec 06 13:38:26 home4 lightningd[669956]: 0x7f3795029d8f __libc_start_call_main
Dec 06 13:38:26 home4 lightningd[669956]:         ../sysdeps/nptl/libc_start_call_main.h:58
Dec 06 13:38:26 home4 lightningd[669956]: 0x7f3795029e3f __libc_start_main_impl
Dec 06 13:38:26 home4 lightningd[669956]:         ../csu/libc-start.c:392
Dec 06 13:38:26 home4 lightningd[669956]: 0x560c8022dc84 ???
Dec 06 13:38:26 home4 lightningd[669956]:         ???:0
Dec 06 13:38:26 home4 lightningd[669956]: 0xffffffffffffffff ???

[tsjk]

Does it work if you specify both --developer and --allow-deprecated-apis=true (in that order - supposedly the latter will override the former's turning the depricated apis off)?

[tsjk]

I still run v23.08.1, but am looking to upgrade. This bug report has put me off a bit, though.

[tsjk]

I couldn't find a way to run v23.11.2 with both clboss and --developer. I'll have to do without the latter for the time being.

[har15eld0n]

Same here, I had to disable clboss to activate the developer mode, and do my thing (running: lightning-cli dev-rescan-outputs) Reactivated the plugin, when I was done...

NOTE: I tried with --allow-deprecated-apis=true and it did not improve the situation...

[Kexkey]

Might have something to do with this: https://github.com/ElementsProject/lightning/blob/v23.11.2/doc/lightning-listpeers.7.md#return-value

- channels (array of objects, optional) deprecated, removal in v23.11:

For some reason, channels in the listpeers response is still there when developer mode is OFF. It has been removed (as announced in the doc) when it is ON, though.

[tsjk]

When developer mode is off deprecated apis are enabled. That's why you have channels there in listpeers. Of course clboss ought be updated to not need deprecated apis, but a plausible workaround for it not working with developer mode was to also enable deprecated apis. This however seems not to work.

[ksedgwic]

It was as simple as clboss didn't know the --developer flag and was exiting w/ usage ... https://github.com/ZmnSCPxj/clboss/pull/216

[ksedgwic]

Fixed