[bug]: channel automatically force closed with low fees and impossible to bumpfees; reimporting fails with "could not derive private key for legacy channel revocation root format: unable to derive private key"

[pony-montana]

Hi, the issue originated in zeus wallet embedded lnd 0.17 (zeus version 0.8.4) but it is still present in zeus with embedded lnd 0.18.0 and lnd desktop 0.18.2.

The channel silently force closed and I didnt noticed nothing in zeus as the channel was still reported as open and online. After three days from closure, I tried do some operations with the channel, like a coop-close, that resulted in silent errors with the channel-visualization remained open and online. I tried then a force-close (that still failed silently, like the coop one) and finally I tried reimporting the wallet with channel backup from scratch. Only at that point I noticed that the forced channel-closure-tx was broadcasted from my peer (olympus) three days prior. The reimported wallet (seed + channel backup) shows finally the channel as closing, but I noticed that the wallet is also slow (like something during restore is went wrong) in all the operations, like it takes 5 minutes only to generate a new onchain receive address and it doesnt permit me to do the bumpfees for the forceclosetx (broadcasted at 1 sat/vbyte).

I tried also to reimport the wallet in lnd desktop and I noticed that the reimporting from backup crashes with log:

unable to unpack chan backup: could not derive private key for legacy channel revocation root format: unable to derive private key

This is also the error showed in zeus lnd logs during importing channel backup.

the channel backup also show in log a corret channel point to restore: prior to fail with the error, it shows:

Restoring ChannelPoint(correctchannelpoint:0) to disk:

So I contacted Olympus to see if the force-close-tx could be fees-bumped from their side; they said that they also cant do the bumpfees on that tx and lnd fails when they try.

What could be happened?

[ViktorTigerstrom]

Hi @pony-montana!

Thanks for reaching out! I'd like to verify some initial troubleshooting, to know how to proceed:

1. In your text, you've written "Only at that point I noticed that the forced channel-closure-tx was broadcasted from my peer (olympus) three days prior." My interpretation of this is that the channel was force closed by your channel peer, and not by your node. Is that correct?
2. Can you view the force closure tx in block explorers, such as mempool.space, or does explorers state that the transaction can't be found? If you can view the transaction: Is there another transaction that is also spending your output of the force closure tx present?
3. Do you have access to any instance of lncli for the wallet? If yes, does lncli wallet listsweeps show any output related to the force-closure?

[pony-montana]

Hi, thanks for reply! :)

1. yes, the channel was closed by peer, not by my node. He say that it wasnt closed intentionally, but some bug accurred and the force-close-tx was broadcasted automatically by his lnd (I think is the last stable 0.17.x version).
2. I can view the force closure tx in block explorers and I dont see any other tx spending the output. I dont see any reference to the closing tx in zeus or lnd, I see the channel-point as the opening tx of the channel, and from here I find the closing tx navigating the block explorer.
3. I'll try as soon as possible, but lnd crashed when I try to reimport the channel backup, it autoshutdown and didnt shows any channel closing.

[ViktorTigerstrom]

Ok thanks for the info!

Given that, the preferred and best alternative would be that you get the channel backup to import correctly. Lnd should then proceed to sweep the force-closure transaction with another transaction with a high enough fee to make both force-closure + the new sweep transaction confirm. If the new sweep transaction gets published with too low fee, lnd can be used to bump the fee for the sweep transaction (i.e. not the force closure transaction).

If you cannot get the channel backup to import correctly, there is an alternative tool you can use to recover your funds called chantools. Note that this isn't the preferred alternative, and it's always better if you can get the channel backup to work properly with lnd. For chantools, there is command called sweepremoteclosed which can be used for your case. Using chantools will require your seed, but using the command you can sweep the funds to an address of your choosing (doesn't need to be an address of the lnd wallet, and shouldn't be either if your having trouble restoring the lnd instance): https://github.com/lightninglabs/chantools/blob/master/doc/chantools_sweepremoteclosed.md

IMPORTANT: if you need to resort to using chantools, you need to specify a feerate that's high enough to make both the force closure + the sweep transaction confirm. Therefore, choose a high fee rate. If you need to resort to using chantools, we can guide you in how to do so.

[ViktorTigerstrom]

Hi again @pony-montana, I just thought of one more option: You can also try to rescan the wallet through Zeus and see if that helps. If you go to: Embedded node > Advanced > Rescan wallet, and then restart the app, It's possible that Zeus will find the channel closure after that. It's worth a try!

[pony-montana]

I'm trying reimporting the wallet in lnd, but when I try to restore channel backup I obtain, as always:

[lncli] unable to restore chan backups: rpc error: code = Unknown desc = unable to unpack chan backup: could not derive private key for legacy channel revocation root format: unable to derive private key

this time without an lnd crash, but still it fails importing and "lncli wallet listsweeps" dont show nothing. Also with zeus I tried multiple time to rescan and reimporting, or try reimporting different channel backups, but nothing. The only thing that remain is to try chantools.

[ViktorTigerstrom]

Hi @pony-montana,

Ok thanks for attempting the above! So it seems like chantools is your best option then.

I just realised though that I recommended the incorrect command for you for chantools as the first step in recovering the funds. You can't spend your to_remote output until the force closure transaction is confirmed. Instead you need to bump the fee of the remote force closure by spending the anchor output so that the transaction confirms. Chantools has a command for this as well, it's called: https://github.com/lightninglabs/chantools/blob/master/doc/chantools_pullanchor.md

So you will first need to make the force closure transaction confirm, and then the your to_remote output can be spent.

Your anchor output is one of the outputs in the force closure transaction that has the value of 330 sats.

Just like previously, you need to set a high enough fee rate that it confirms both the force closure and the anchor spend transactions.

[pony-montana]

Hi, thanks a lot! Do I need onchain balance for paying the fees or they will be deducted from the output?

edit: I see the command pullanchor requires a sponsor input, so it is required

[ViktorTigerstrom]

edit: I see the command pullanchor requires a sponsor input, so it is required

Correct :)!

[pony-montana]

Hi I did:

chantools pullanchor --anchoraddr bc1p...(casually selected one of the two address that receive 330 sats in the channel-close-tx) --changeaddr bc1q... --feerate 15 --sponsorinput 12345678:0 (the imput I want to use to bump fees)

the output seems good, no errors:

[INF] CHAN: Found multisig key 387249724792...(never seen the string before) for anchor pk script 48375953afaf..(never seen this string before) [INF] CHAN: Fee 2500 sats of xxx total amount (estimated weight 670) [INF] CHAN: Prepared PSBT follows, please now call 'lncli wallet psbt finalize ' to finalize the transaction, then publish it manually or by using 'lncli wallet publishtx ':

cH3455352...( the psbt)

Now I'm having some problem signin the psbt, I obtain errors, I dont know If its becasue of the excape characters triggering the shell, maybe I can put the psbt in a file? Trying playng with various method to insert the command lncli wallet psbt finalize I obtained different errors:

• [lncli] rpc error: code = Unknown desc = error finalizing PSBT: error finalizing PSBT: PSBT is not finalizable
• I cant find it, but it complained about the string wasnt base64 formatted

EDIT: the string is well elaborated, the first error is caused by something else EDIT2: selecting the other anchor address I obtained an error, so I imagine I used the correct anchor input the first try EDIT3: I imported the wallet in sparrow and finally signed the tx, then I broadcasted with lncli. Still dont know what was the problem with lncli not signin the tx

[pony-montana]

I recovered the funds, thanks a lot for your help and for the fantastic tools you are developing here! This has been a fantastic challenge to learn something new :).

I think the issue can be closed!

I found also that it wasnt possible to sign with lncli the psbt generated with chantools pullanchor because I didnt imported correctly the wallet in lnd...

Trying to make this issue the most useful as possible for who will see it in future, here are summarized the problems that have remained without a solution:

• what triggered the force close by zeus lnd node? (not enough info to answer)
• why I didnt saw the force closure into zeus wallet? (not totally lnd related)
• why lnd crashed when I tryed to reimport the channel backup? What type of corruption could happened? How is it related to the bug experienced by the olympus (the peer) node? (probably even here, not enough info to answer)
• why olympus node failed to do bumpfees of the force-close-tx? This is totally unrelated to possible backup corruptions happened in my node.

[kaloudis]

Thanks for the support @ViktorTigerstrom

I wonder what is causing the backup recovery to fail here

[guggero]

Do you still have the backup file that couldn't be recovered? If yes, could you run it through chantools dumpbackup (requires you to enter the seed to decrypt the backup) and post the output here? Or send it as a DM to one of us (we're all on our Slack server).

[pony-montana]

I made a disaster, I was operating in a virtual environment and I nuked it after the recovery succeded, probably I lost the seed. My really bad, I could have thinked to save the seed as it would still be useful for understanding the problem, I'm so sorry. I'll try to see if I'm able to recover it and if I made some kind of backup along the way...

[guggero]

Okay, no problem. I'm going to close the issue in the meantime, but feel free to post here again if you are able to recover the seed and backup file.