Optimize lnd neutrino sync

[kilrau]

Currently, lndbtc/lndltc sync takes some 10-15 minutes on simnet, about 3h on mainnet.

What can be optimized:

• [x] enable routing.assumechanvalid=1 to speed up graph validation, see https://github.com/lightningnetwork/lnd/blob/master/routing/conf_experimental.go. Not sure if we need that since this seems to be rather saving CPU and we would need to find an alternative way to know about closed channels.
• [ ] above needs an alternative way to learn about closed channels. In Breez, this is done in a process which launches lnd as a library. See https://github.com/breez/breez/blob/7c1d567234be036a635d787829fad46de958e9c7/closedchannels/job.go which downloads a list of closed channels for each 100blocks. The data itself is prepared by a c-lightning node with a small patch to export the closed channels into a db
• [ ] sync from latest checkpoint only. Currently, lndbtc, lndltc sync & verify filter headers for the whole chain. How it's done: https://github.com/breez/breez/pull/97/files

[michael1011]

I understand the reason for which mobile wallets do these optimizations but we are not a mobile wallet running on a low powered Android device with limited bandwidth.

I would avoid adding even more security assumptions to our light client setup. Checkpoints would sound reasonable if we really really (really) want to have our clients synced in a couple seconds which is not necessary I think. 10 minutes of syncing is not too bad considering that you have a fully featured DEX afterwards.

routing.assumechanvalid=1 is not worth the effort for us right now.

What we can do for now is also use our own nodes to provide the Neutrino filters for the Bitcoin chain. That would mitigate the problem of the banned peers and timeouts until there are more reliable BTCD nodes on the network.

@kilrau

[kilrau]

Agree on leaving routing.assumechanvalid=1 aside for now and try checkpoints + reliable nodes.

[kilrau]

My nodes are up:

btc.kilrau.com 8333
btc.kilrau.com 18333
ltc.kilrau.com 9333
ltc.kilrau.com 19335

[kilrau]

Did a lnd 0.10 test with neutrino today too (mainnet+testnet) connecting to my own btc.kilrau.com full nodes. Result=Neutrino processes filters from genesis to tip with about 4000 blocks/minute.

2020-05-18 17:14:25.357 [INF] BTCN: Processed 2000 blocks in the last 25.23s (height 150445, 2011-10-24 12:55:09 +0000 UTC)
2020-05-18 17:16:42.555 [INF] BTCN: Verified 2000 filter headers in the last 28.83s (height 150001, 2011-10-20 13:51:45 +0000 UTC)

[kilrau]

Since things are working fine and Neutrino syncs in 15-30 mins on mainnet and as @michael1011 described we would want to avoid adding more security assumptions, we will leave things as is for now and only revisit this if syncing Neutrino becomes a problem. I'll leave this issue open though.