Neutrino 🔁 Bitcoin Node Automatic Switching

mayankchhabra commented 4 years ago

Issue

The user cannot start using his lightning or bitcoin wallet until bitcoind has fully synchronized. This adds a lot of friction due to the long delay, creating a very frustrating experience.
If bitcoind lags behind for some reason, such as when the user had his node turned off for a few days/weeks, the same problem reoccurs.

Solution

Neutrino filters can solve this problem. We need an automatic, recurring way to switch lnd's bitcoin mode to neutrino from bitcoind as soon as bitcoind lags behind by a certain number of blocks from its best header.

Ideas

Perhaps it's a good idea to create a new service that runs 24x7 via a container called lnd-switch that uses the following environment variables:

BITCOIN_HOST: "0.0.0.0"
BITCOIN_RPC_PORT: "8332"
BITCOIN_RPC_USER: "umbrel"
BITCOIN_RPC_PASSWORD: "1234567890"
LND_CONFIG_FILE: "/home/umbrel/lnd/lnd.conf"
SWITCH_TO_NEUTRINO: "144" (switch to neutrino when bitcoind is *at least* 144 blocks behind)
SWITCH_TO_BITCOIND: "3" (switch to bitcoind when bitcoind is *at max* 3 blocks behind)

Considering the above example variables, lnd-switch will update the bitcoin node in LND_CONFIG_FILE from bitcoin.node=bitcoind to bitcoin.node=neutrino as soon as bitcoind's headers - blocks >= 144 and update it from bitcoin.node=neutrino to bitcoin.node=bitcoind as soon as bitcoind's headers - blocks <= 3. It also needs a way to restart the lnd container after every config file change.

I'm not sure if this is the right approach or not. If it is, then lnd-switch can be a tiny FOSS project on its own that (I think) many more people will find useful.

cc @nolim1t @meeDamian

mayankchhabra commented 4 years ago

Instead of creating a new docker service, perhaps this should be handled by https://github.com/getumbrel/umbrel-manager

nolim1t commented 4 years ago

lets make it $HOST/lnd/lnd.conf or /mnt/data/lnd/lnd.conf

nolim1t commented 4 years ago

can be a docker service for sure and restart: on-error

mayankchhabra commented 4 years ago

lets make it $HOST/lnd/lnd.conf or /mnt/data/lnd/lnd.conf

I didn't fully understand that.. can you please elaborate a bit?

Also, since $HOST/lnd/lnd.conf is symlinked to /mnt/data/lnd/lnd.conf, the lnd-switch can make changes directly to $HOST/lnd/lnd.conf

lukechilds commented 4 years ago

Concept ACK.

Would love to see this, it's a feature that I hear a lot of people talk about but AFAIK no one has implemented yet, would be great if Umbrel was the first.

nolim1t commented 4 years ago

lets make it $HOST/lnd/lnd.conf or /mnt/data/lnd/lnd.conf

I didn't fully understand that.. can you please elaborate a bit?

Also, since $HOST/lnd/lnd.conf is symlinked to /mnt/data/lnd/lnd.conf, the lnd-switch can make changes directly to $HOST/lnd/lnd.conf

${HOME}

nolim1t commented 4 years ago

it would need to keep running too and be able to switch back and forth too.

Say bitcoind is corrupted it should switch back and forth

nolim1t commented 4 years ago

Will attempt to do this. Probably a bash looping service similar to the unlock script

mayankchhabra commented 4 years ago

Awesome! Looking forward. It really is one of the last few missing pieces before we can launch.

nolim1t commented 4 years ago

going to have a stab at this now.. it will need a lot of testing

nolim1t commented 4 years ago

sample code which can do it

#!/bin/bash -e

PASSWORD=`cat $PWD/secrets/rpcpass.txt`

INFO=`curl --user lncm:$PASSWORD --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getblockchaininfo", "params": [] }' http://10.254.2.2:18332 2>/dev/null`

HEADERS=`echo $INFO | jq .result.headers`
BLOCKS=`echo $INFO | jq .result.blocks`

if [ $HEADERS -eq $BLOCKS ]; then
    echo "Switching over from bitcoind to neutrino"
    sed 's/bitcoin.node\=neutrino/bitcoin.node\=bitcoind/g; ' lnd/lnd.conf
fi

mayankchhabra commented 4 years ago

Looking good

nolim1t commented 4 years ago

https://hub.docker.com/repository/docker/lncm/neutrino-switcher

igorbastosib commented 9 months ago

I know it is an old discussion here, I couldn't find anything new about the subject.

Is it still possible to use 100% LN + Neutrino (or any way to connect to a Bitcoin node on another device)?

I mean, I have a Device (not Umbrel) with a synched BitcoinCoreNode running, I'd like to prepare several Umbrel devices with LN and want to link them to my external BitcoinCoreNode (external just because it is not on the same device, but all on the same Network)

If yes, could someone please share how?

getumbrel / umbrel