After 1.7 LiT installation, missing Pool account/funds

[ghost]

I had pool with an active account and funds on 1.6.X. After upgrading to 1.7 and installing LiT, I no longer seem to have my existing pool account. I have active leases and funds so I'm a bit lost on what I'm supposed to do to recover those.

It's a non trivial amount so any advise would be great.

[rootzoll]

LIT is a new bundle of pool - its not using the old data. But you can reactivate the old pool single instance also which should have your old data with /home/admin/config.scripts/bonus.pool.sh on on terminal - funds will only show in the terminal version, not within LIT WebUI. After install see info on /home/admin/config.scripts/bonus.pool.sh menu.

[ghost]

Doesn't work.

pool@raspberrypi:~ $ pool getinfo
[pool] open /home/pool/.pool/mainnet/tls.cert: no such file or directory

[ghost]

More problems.

Reboot seemed to allow me to call pool getinfo but now with a different problem

pool@raspberrypi:~ $ pool getinfo
[pool] rpc error: code = Unimplemented desc = unknown method GetInfo for service poolrpc.Trader

but I can run some other stuff like accounts:

pool accounts list
{
    "accounts": []
}

However seems like my pool info was deleted.

reinstalling lit doesn't work because it doesn't remove the group properly.

admin@raspberrypi:~ $  /home/admin/config.scripts/bonus.lit.sh on
making sure the lit service is not running
# INSTALL LIGHTNING TERMINAL
adduser: The group `lit' already exists.

Trying to recover funds with pool or lit-pool results in

pool@raspberrypi:~ $ pool accounts recover
[pool] rpc error: code = Unknown desc = error performing recovery: could not recover account XXX: error during authentication, before sending subscribe: rpc error: code = Internal desc = payment required

Despite being well connected enough.

I guess I'll wipe my device and try again?

Some suggestions of things to fix so others don't lose funds like I have.

1. Don't install LiT if there's an existing pool installation.
2. Warn users that a pool uninstall will result in a loss of funds (your LiT installation seems to be deleting my existing pool data, but if you fix 1, then warn)
3. Uninstall LiT properly so that it may be reinstalled again

I have no idea why a well connected node is having payment problems to pool, I can only imagine something screwed up with the install in general.

[rootzoll]

@openoms can you take a look

[mbombonato]

I can also confirm that my attempt of reinstalling LiT was unsuccessful.

admin@raspberrypi:~ $  /home/admin/config.scripts/bonus.lit.sh on
making sure the lit service is not running
# INSTALL LIGHTNING TERMINAL
adduser: The group `lit' already exists.

After removing the group manually I was able to reinstall it using the script. I think the uninstall script is not removing the LiT group. Let me know if I should open a proper issue for this. I'm just giving my feedback for now.

Thanks for your awesome work!

[openoms]

@mbombonato to reinstall first uncheck in SERVICES or use: /home/admin/config.scripts/bonus.lit.sh off that will run: sudo userdel -rf lit https://github.com/rootzoll/raspiblitz/blob/be19c565cb5fb570abcd7cf6ad83cc5e4b840520/home.admin/config.scripts/bonus.lit.sh#L360 which removes the user. can use this step manually.

Uninstalling the lit user does nor delete data because it is symlinked from the disk.

@rootzoll bonus.loop.sh and bonus.lit.sh are using the same data directory: /mnt/hdd/app-data/.pool/ symlinked from the disk. https://github.com/rootzoll/raspiblitz/blob/be19c565cb5fb570abcd7cf6ad83cc5e4b840520/home.admin/config.scripts/bonus.lit.sh#L146

@AnthonyRonning Likewise you pool data should be in: /mnt/hdd/app-data/.pool/ and should be appearing on a reinstall between scripts as that directory is not deleted in any script. To reach the Pool CLI while having lit installed use the alias:
lit-pool can you please try reinstalling LIT again from the menu? and check the contents:

$ ls -la /mnt/hdd/app-data/.pool/
total 16
drwxr-xr-x  4 lit  lit  4096 Oct 10  2020 .
drwxr-xr-x 15 root root 4096 Mar 28 11:02 ..
drwxr-xr-x  3 lit  lit  4096 Oct 10  2020 logs
drwxr-xr-x  2 lit  lit  4096 Oct 11  2020 mainnet

confirm the symlink:

$ ls -la /home/lit/.pool
lrwxrwxrwx 1 root root 24 Apr 18 17:52 /home/lit/.pool -> /mnt/hdd/app-data/.pool/

[openoms]

@AnthonyRonning just noting for the peace of mind, that even if the pool account has been somehow deleted the funds in the pool multisig will be returned automatically to the onchain part of the lnd wallet after the Pool account expires.

Checked that the bonus.pool.sh was using the /mnt/hdd/app-data/.pool directory to store the data as well. https://github.com/rootzoll/raspiblitz/blob/755b7460ca7d8f6aa614748e534ec0941fc891ea/home.admin/config.scripts/bonus.pool.sh#L55

Is that possible that you have installed Pool manually or with some early, experimental version of the script?

[ghost]

even if the pool account has been somehow deleted the funds in the pool multisig will be returned automatically to the onchain part of the lnd wallet after the Pool account expires.

I'm chatting with guggero on the lightning labs slack and he says that's not the case. A manual close is required to return the funds back.

I've done reinstalls (after manualing deleting the lit group) and even reinstalled the OS again. Still showing no auth, no accounts, etc.. Confirmed ls -la /home/lit/.pool is where my data is.

I only started using Pool with the last version, I would not have done anything manual or experimental. At this point I consider my data a goner and trying to go through recovery mechanisms but still can't recover funds with pool.

[openoms]

@AnthonyRonning
Is ls -la /home/lit/.pool not a symlink?

Do you have anything on the disk in: /mnt/hdd/app-data/.pool ?

Did you overwrite your SDcard on update? Did you happen to have the old one?

[ghost]

It is

lit@raspberrypi:~ $ ls -la /home/lit/.pool
lrwxrwxrwx 1 root root 24 May  3 20:34 /home/lit/.pool -> /mnt/hdd/app-data/.pool/

I have files there, but at this point not sure if it was the original files or the new ones.

lit@raspberrypi:/mnt/hdd/app-data/.pool/mainnet $ ls
macaroons.db  pool.db  pool.macaroon

I had my old SDcard until I overwrote it with my 2nd installation attempt today.

[openoms]

So the old data is there. There was no command in the scripts which could have deleted it. Can you check when the pool.db was last modified (ls -la /mnt/hdd/app-data/.pool/mainnet/pool.db )?

What does: lit-pool getinfo show?

Do you have at least one active channel on the node with which you can pay the LSAT token?

(The SDcard was unlikely having anything since the data was symlinked from the disk in v1.6.3 as well).

[ghost]

Showed that it was last modified today. I'm pretty sure I moved that folder to a backup folder today during reinstalls but not sure if I moved the right folder or a symlink tbh.

I was able to recover my funds though. There is a bug in the poold version that is included in lit where it wouldn't allow recovery due to an LSAT bug, AFAIK.

I uninstalled lit, turned on the standalone pool, ran a git fetch & git pull and installed the master copy of pool. Was able to recover funds then.

Thanks for your help! Not sure what happened with me in the first place with accounts not loading in, but at least I'm squared away now.

FYI, I think the lit uninstall issue was from RTL being added as a user to the lit group but not being removed from the group before groupdel runs (thought I caught a glimpse as uninstall ran once).

Unless you want to leave this open for anything, feel free to close. Thanks again!

[openoms]

Thank you @AnthonyRonning it was a good exercise. Will try to reproduce and fix this uninstall bug before closing this.

[ghost]

Apparently the latest pool master won't work for creating orders, so I had to downgrade pool.

I copied my pool data (/mnt/hdd/app-data/.pool/mainnet/) in a back up folder, uninstalled pool standalone, installed lit, and my lsat and key/cert had been removed. Accounts were missing again.

I put the backed up files back, restarted lit, and was able to have my accounts show up again. Just an FYI. Either my machine is just messed up a bit or these are some reproduction steps you might be able to try.

[ramontayag]

I'm running into the same issue. I don't have my old SD anymore. I have installed lit on the new SD, tried turning on the old pool but unable to getinfo (I get the poolrpc.Trader error). pool listauth does not show any LSAT.

@AnthonyRonning how did you install the master version of pool? You just ran make install?

[ghost]

I also had that error, I think it was related to incompatible macaroons with the new version.

Move your pool dir first (/mnt/hdd/app-data/.pool/) to have as a backup just in case. You should then, for safe measures, uninstall lit and reinstall pool, hopefully you get a good macaroon again and can at least getinfo.

After installing pool again, go to the pool directory (I think it's /home/pool/pool) and do a git pull. Afterwards, run this line

https://github.com/rootzoll/raspiblitz/blob/v1.7/home.admin/config.scripts/bonus.pool.sh#L207

Afterwards:

pool

pool accounts recover

It should hopefully find your account. If that's true, you can either close out your account if it's expired, or keep the account open.

If you want to keep the account open and create orders from it, you'll need to downgrade again. Back up your pool data again, uninstall pool, install lit, and put your pool data back. Hopefully that works for you, this is pretty much the entire process I went through and was able to recover my funds and use the integrated lit.

[ramontayag]

Unfortunately, I'm unable to successfully call getinfo. I threw out a question in Slack about recovering a pool account, and he said this (posting for others):

I think I lost my pool data. I still have my lightning node running with the same public key as before though. If I pay for a new LSAT (I’m assuming this was lost), can I recover my pool account?

Guggero replied:

yes, but because of an LSAT related problem that was fixed recently, you need to build and compile the master branch of Pool from source for the account recovery. And then you need to downgrade again to the latest tagged version (v0.4.4-alpha) because we first need to deploy something on the server for you to be able to submit orders with the master version. a bit tricky, sorry. we're blocked on tagging a new version by a feature that's still in development

I can't find a file lsat.token anywhere, so it looks like I have to get a new LSAT token and begin account recovery.

[openoms]

I think I found the bug. If the user has used the standalone Pool before it is installed after LiT during the recovery process when changing the SDcard.

Check for the owner of the lsat.token with:

ls -la /mnt/hdd/app-data/.pool/mainnet/lsat.token it should be:

-rw-r--r-- 1 lit lit 57 May  5 09:34 /mnt/hdd/app-data/.pool/mainnet/lsat.token

if it is:

-rw-r--r-- 1 pool pool 57 May  5 09:34 /mnt/hdd/app-data/.pool/mainnet/lsat.token

the standalone Pool is running in the background too.

Fix with running the commands:

# remove the standalone Pool
config.scripts/bonus.pool.sh off
# fix the lsat token permission
sudo chown lit:lit -R /mnt/hdd/app-data/.pool
# restart LiT
sudo systemctl restart litd

PR #2276 wil fix this for v1.7.1.

[openoms]

I found the bigger issue also. Unfortunately https://github.com/rootzoll/raspiblitz/pull/1957 introduced the command which deletes the Pool data when running: config.scripts/bonus.pool.sh off

This results in the Pool data being deleted when installing LiT while the standalone Pool is installed.

Fixing this as well in #2276

@rootzoll could this be a trigger for v1.7.1 release soon?

[openoms]

In case the problem already happened (Pool data missing) got a general LSAT token from guggero which can be downloaded from: https://oms.keybase.pub/lsat.token

if you want, you can give this token to anyone who has that problem. they should only use it for recovering/closing their account though and then start afresh (remove the token and create a new account) that way people don't need to update to master to get a new token.

The procedure for people with the problem would be:

1. Place the token into /mnt/hdd/app-data/.pool/mainnet/lsat.token

   # make sure the directory exists and owned by lit
   sudo -u lit mkdir -p /mnt/hdd/app-data/.pool/mainnet/
   sudo chown lit:lit -R /mnt/hdd/app-data/.pool
   # download the token
   sudo -u lit wget -O /mnt/hdd/app-data/.pool/mainnet/lsat.token https://oms.keybase.pub/lsat.token

2. Start LiT, then use lit-pool accounts recover

   sudo systemctl restart litd
   lit
   lit-pool accounts recover

3. Check account is back with:

   lit-pool accounts list --show_archived

[openoms]

If you have a Pool account and not installed LiT yet get rid of the problematic line with:

sed -i 's#sudo rm -r /mnt/hdd/app-data/.pool##' /home/admin/config.scripts/bonus.pool.sh

[ramontayag]

Thanks @openoms! Your instructions worked perfectly for me.

[rootzoll]

@openoms is anything to take care on this still for v1.7.1?

[openoms]

Fixed and tested, can be closed.