search

lightningnetwork / lnd

Lightning Network Daemon ⚡️
MIT License
7.67k stars 2.07k forks source link

LND error #4743

Closed PandamineHub closed 3 years ago

PandamineHub commented 3 years ago

Background

Hello everybody,

could you please help me? MyNode has been working for few days, I have set up my lightning wallet and open few channels. After one restart of my raspberry 4 with MyNode, LND is still in status "Opening DB" and there is showing below error. It looks like there is missing certificate, but I am not sure how it could be possible when it worked before.Could you please help me?

Your environment

Steps to reproduce

Lnd not working, still trying to open, but cannot find certificate and I did not delete it and recreation is not working.

Expected behaviour

lnd should work normaly

Actual behaviour

Nov 03 01:22:16 myNode systemd[1]: loopd.service: Failed with result 'exit-code'. Nov 03 01:22:16 myNode systemd[1]: loopd.service: Main process exited, code=exited, status=1/FAILURE Nov 03 01:22:16 myNode loopd[9437]: loopd exited with an error: open /home/bitcoin/.lnd/tls.cert: no such file or directory Nov 03 01:22:16 myNode loopd[9437]: 2020-11-03 01:22:16.481 [INF] LNDC: Creating lnd connection to localhost:10009 Nov 03 01:22:16 myNode loopd[9437]: 2020-11-03 01:22:16.479 [INF] LOOPD: Version: 0.8.1-beta commit= Nov 03 01:22:16 myNode systemd[1]: Started loopd. Nov 03 01:22:16 myNode loopd[9425]: [lncli] open /mnt/hdd/mynode/lnd/tls.cert: no such file or directory Nov 03 01:22:16 myNode loopd[9425]: Checking if LND is synced... Nov 03 01:22:16 myNode loopd[9424]: Not shutting down! Nov 03 01:22:16 myNode loopd[9424]: Checking if device is shutting down... Nov 03 01:22:16 myNode systemd[1]: Starting loopd...

guggero commented 3 years ago

The log you posted is from the loopd service. Can you please post the logs from lnd?

PandamineHub commented 3 years ago

ou

-- Logs begin at Tue 2020-11-03 01:48:47 CST, end at Tue 2020-11-03 05:18:22 CST. -- Nov 03 05:18:21 myNode systemd[1]: Failed to start LND Lightning Daemon. Nov 03 05:18:21 myNode systemd[1]: lnd.service: Failed with result 'exit-code'. Nov 03 05:18:21 myNode systemd[1]: lnd.service: Start request repeated too quickly. Nov 03 05:18:21 myNode systemd[1]: Stopped LND Lightning Daemon. Nov 03 05:18:21 myNode systemd[1]: lnd.service: Scheduled restart job, restart counter is at 5. Nov 03 05:18:21 myNode systemd[1]: lnd.service: Service RestartSec=1s expired, scheduling restart. Nov 03 05:18:20 myNode systemd[1]: lnd.service: Failed with result 'exit-code'. Nov 03 05:18:20 myNode systemd[1]: lnd.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Nov 03 05:18:20 myNode lnd[11335]: go.etcd.io/bbolt@v1.3.5-0.20200615073812-232d8fc87f50/db.go:1001 +0x10c Nov 03 05:18:20 myNode lnd[11335]: created by go.etcd.io/bbolt.(DB).freepages Nov 03 05:18:20 myNode lnd[11335]: go.etcd.io/bbolt@v1.3.5-0.20200615073812-232d8fc87f50/db.go:1003 +0xcc Nov 03 05:18:20 myNode lnd[11335]: go.etcd.io/bbolt.(DB).freepages.func2(0x3b2e480) Nov 03 05:18:20 myNode lnd[11335]: goroutine 38 [running]: Nov 03 05:18:20 myNode lnd[11335]: panic: freepages: failed to get all reachable pages (page 5835: multiple references) Nov 03 05:18:19 myNode lnd[11335]: 2020-11-03 05:18:19.590 [INF] LTND: Opening bbolt database, sync_freelist=false Nov 03 05:18:19 myNode lnd[11335]: 2020-11-03 05:18:19.590 [INF] LTND: Opening the main database, this might take a few minutes... Nov 03 05:18:19 myNode lnd[11335]: 2020-11-03 05:18:19.590 [INF] LTND: Active chain: Bitcoin (network=mainnet) Nov 03 05:18:19 myNode lnd[11335]: 2020-11-03 05:18:19.590 [INF] LTND: Version: 0.11.0-beta commit=v0.11.0-beta, build=production, logging=default Nov 03 05:18:19 myNode lnd[11335]: Automatically obtained bitcoind's RPC credentials Nov 03 05:18:19 myNode lnd[11335]: Attempting automatic RPC configuration to bitcoind Nov 03 05:18:19 myNode systemd[1]: Started LND Lightning Daemon. Nov 03 05:18:19 myNode lnd[11333]: + exit 0 Nov 03 05:18:19 myNode lnd[11333]: } Nov 03 05:18:19 myNode lnd[11333]: "warnings": "" Nov 03 05:18:19 myNode lnd[11333]: }, Nov 03 05:18:19 myNode lnd[11333]: } Nov 03 05:18:19 myNode lnd[11333]: "height": 481824 Nov 03 05:18:19 myNode lnd[11333]: "active": true, Nov 03 05:18:19 myNode lnd[11333]: "type": "buried", Nov 03 05:18:19 myNode lnd[11333]: "segwit": { Nov 03 05:18:19 myNode lnd[11333]: }, Nov 03 05:18:19 myNode lnd[11333]: "height": 419328 Nov 03 05:18:19 myNode lnd[11333]: "active": true, Nov 03 05:18:19 myNode lnd[11333]: "type": "buried", Nov 03 05:18:19 myNode lnd[11333]: "csv": { Nov 03 05:18:19 myNode lnd[11333]: }, Nov 03 05:18:19 myNode lnd[11333]: "height": 388381 Nov 03 05:18:19 myNode lnd[11333]: "active": true, Nov 03 05:18:19 myNode lnd[11333]: "type": "buried", Nov 03 05:18:19 myNode lnd[11333]: "bip65": { Nov 03 05:18:19 myNode lnd[11333]: }, Nov 03 05:18:19 myNode lnd[11333]: "height": 363725 Nov 03 05:18:19 myNode lnd[11333]: "active": true, Nov 03 05:18:19 myNode lnd[11333]: "type": "buried", Nov 03 05:18:19 myNode lnd[11333]: "bip66": { Nov 03 05:18:19 myNode lnd[11333]: }, Nov 03 05:18:19 myNode lnd[11333]: "height": 227931 Nov 03 05:18:19 myNode lnd[11333]: "active": true, Nov 03 05:18:19 myNode lnd[11333]: "type": "buried", Nov 03 05:18:19 myNode lnd[11333]: "bip34": { Nov 03 05:18:19 myNode lnd[11333]: "softforks": { Nov 03 05:18:19 myNode lnd[11333]: "pruned": false, Nov 03 05:18:19 myNode lnd[11333]: "size_on_disk": 349660099137, Nov 03 05:18:19 myNode lnd[11333]: "chainwork": "00000000000000000000000000000000000000001559bad44d2bbd51059c092b", Nov 03 05:18:19 myNode lnd[11333]: "initialblockdownload": false, Nov 03 05:18:19 myNode lnd[11333]: "verificationprogress": 0.9999984586486783, Nov 03 05:18:19 myNode lnd[11333]: "mediantime": 1604399746, Nov 03 05:18:19 myNode lnd[11333]: "difficulty": 16787779609932.66, Nov 03 05:18:19 myNode lnd[11333]: "bestblockhash": "00000000000000000001400d44cf50c616097db3d642e70c700dcff3ff6cd11b", Nov 03 05:18:19 myNode lnd[11333]: "headers": 655218, Nov 03 05:18:19 myNode lnd[11333]: "blocks": 655218, Nov 03 05:18:19 myNode lnd[11333]: "chain": "main", Nov 03 05:18:19 myNode lnd[11333]: { Nov 03 05:18:19 myNode lnd[11333]: + bitcoin-cli -datadir=/mnt/hdd/mynode/bitcoin -rpcwait getblockchaininfo Nov 03 05:18:19 myNode lnd[11333]: + '[' '!' -f /mnt/hdd/mynode/.mynode_bitcoind_synced ']' Nov 03 05:18:19 myNode lnd[11333]: Checking if Bitcoin is synced... Nov 03 05:18:19 myNode lnd[11333]: + echo 'Checking if Bitcoin is synced...' Nov 03 05:18:19 myNode lnd[11333]: + set -e Nov 03 05:18:19 myNode lnd[11330]: Not shutting down! Nov 03 05:18:19 myNode lnd[11330]: Checking if device is shutting down... Nov 03 05:18:19 myNode systemd[1]: Starting LND Lightning Daemon... Nov 03 05:18:19 myNode systemd[1]: Stopped LND Lightning Daemon. Nov 03 05:18:19 myNode systemd[1]: lnd.service: Scheduled restart job, restart counter is at 4. Nov 03 05:18:19 myNode systemd[1]: lnd.service: Service RestartSec=1s expired, scheduling restart. Nov 03 05:18:18 myNode systemd[1]: lnd.service: Failed with result 'exit-code'. Nov 03 05:18:18 myNode systemd[1]: lnd.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Nov 03 05:18:18 myNode lnd[11316]: go.etcd.io/bbolt@v1.3.5-0.20200615073812-232d8fc87f50/db.go:1001 +0x10c Nov 03 05:18:18 myNode lnd[11316]: created by go.etcd.io/bbolt.(DB).freepages Nov 03 05:18:18 myNode lnd[11316]: go.etcd.io/bbolt@v1.3.5-0.20200615073812-232d8fc87f50/db.go:1003 +0xcc Nov 03 05:18:18 myNode lnd[11316]: go.etcd.io/bbolt.(DB).freepages.func2(0x352a140) Nov 03 05:18:18 myNode lnd[11316]: goroutine 36 [running]: Nov 03 05:18:18 myNode lnd[11316]: panic: freepages: failed to get all reachable pages (page 5835: multiple references) Nov 03 05:18:17 myNode lnd[11316]: 2020-11-03 05:18:17.917 [INF] LTND: Opening bbolt database, sync_freelist=false Nov 03 05:18:17 myNode lnd[11316]: 2020-11-03 05:18:17.916 [INF] LTND: Opening the main database, this might take a few minutes... Nov 03 05:18:17 myNode lnd[11316]: 2020-11-03 05:18:17.898 [INF] LTND: Active chain: Bitcoin (network=mainnet) Nov 03 05:18:17 myNode lnd[11316]: 2020-11-03 05:18:17.897 [INF] LTND: Version: 0.11.0-beta commit=v0.11.0-beta, build=production, logging=default Nov 03 05:18:17 myNode lnd[11316]: Automatically obtained bitcoind's RPC credentials Nov 03 05:18:17 myNode lnd[11316]: Attempting automatic RPC configuration to bitcoind Nov 03 05:18:17 myNode lnd[11317]: abc > /mnt/hdd/mynode/lnd/data/chain/bitcoin/mainnet/poke_login Nov 03 05:18:17 myNode systemd[1]: Started LND Lightning Daemon.

guggero commented 3 years ago

panic: freepages: failed to get all reachable pages (page 5835: multiple references)

This means your channel database has experienced data corruption. Was there an unclean shutdown of the node, for example because of a power outage or accidental unplug?

You can try to create a copy of the database in safe mode with the chantools compactdb command: https://github.com/guggero/chantools#compactdb

Make sure you download the binary for the correct architecture.

If you don't see any errors when running that command, you can continue to operate your node safely (after replacing the original channel.db file with the compacted one). If there are errors, it's probably safer to execute the disaster recovery scenario even though that will close all channels and incur on-chain fees.

PandamineHub commented 3 years ago

Hi guggero. Thank you very much for your support. I did following:

Now I am trying run compactdb and I have following error:

admin@myNode:~ $ chantools compactdb --sourcedb /mnt/hdd/mynode/lnd/data/graph/mainnet/channel.db --destdb /mnt/hdd/results/compacted.db -bash: chantools: command not found admin@myNode:~ $

image

Could you please help me what I did wrong?

Thank you very much!

guggero commented 3 years ago

If you have the chantools command in the current directory you are at (and not in your PATH environment variable), you have to run it with ./chantools compactdb ......

guggero commented 3 years ago

@PandamineHub what's the status on this issue? Were you able to fix your channel DB with the compactdb command?

darrenahunter commented 3 years ago

hi @guggero - I also have the problem described above I am also trying your compactdb method however having the same issue as @PandamineHub it appears. I have installed go and chantools and run 'make install' - all when well however it looks like to me that the cmd chantools is created in a directory go/bin at the same level as chantools directory - you can see this in the screenshot above. so we are not able to run the chantools command as described. if I move the chantools cmd to chantools directory (ie. mv ../go/bin/chantools chantools ) then I can run ./chantools compactdb from within chantools directory I think you should update your readme for chantools to do this - or update the make file

many thanks Darren

guggero commented 3 years ago

I'll update the readme, yes. I provide precompiled binaries in the GitHub release section. I should just link to those.

darrenahunter commented 3 years ago

Hi Olivier

thanks for reply

I would also let you know that compactdb didn’t fix my issue. the chantools compactdb command returned errors and failed - sorry I didn’t capture the screen - but it was basically the ‘bucket already exists’ type error.

Luckily I didn’t have much in the way of channels or sats to worry about - so I deleted channels.db and LND restarted ok

thanks

On 23 Dec 2020, at 11:17 pm, Oliver Gugger notifications@github.com wrote:

I'll update the readme, yes. I provide precompiled binaries in the GitHub release section. I should just link to those.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lightningnetwork/lnd/issues/4743#issuecomment-750245583, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2RIAZLBUVQKLX6CBL53DLSWHNUFANCNFSM4TIMNGHA.

guggero commented 3 years ago

Do you have a channel.backup file from before you deleted the channel.db? Then you can at least close the channels and get the funds back if you run lncli restorechanbackup!

darrenahunter commented 3 years ago

Many Thanks Yes I do I will issue the cmd and see Thanks again

On Thu, 24 Dec 2020, 6:37 pm Oliver Gugger, notifications@github.com wrote:

Do you have a channel.backup file from before you deleted the channel.db? Then you can at least close the channels and get the funds back if you run lncli restorechanbackup!

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/lightningnetwork/lnd/issues/4743#issuecomment-750790068, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2RIA2P55UIWOV6FDD5HKDSWLVURANCNFSM4TIMNGHA .

Roasbeef commented 3 years ago

Closing due to inactivity.