Enhanced Warm Up Check

[reliveyy]

Describe the bug The "xucli create" step will fail with below 4 cases constantly over 10 times

• Error: 13 INTERNAL: could not initialize lnd-LTC/BTC: 14 UNAVAILABLE: failed to connect to all addresses
• Error: 14 UNAVAILABLE: lnd-LTC/BTC is Disconnected

❯ ~/xud.sh
1) Simnet
2) Testnet
3) Mainnet
Please choose the network: 1
🚀 Launching simnet environment
🌍 Checking for updates...
- Container simnet_lndbtc_1: missing
- Container simnet_lndltc_1: missing
- Container simnet_connext_1: missing
- Container simnet_xud_1: missing
Creating simnet_lndbtc_1...
Creating simnet_lndltc_1...
Creating simnet_connext_1...
Creating simnet_xud_1...

🏃 Warming up...

Do you want to create a new xud environment or restore an existing one?
1) Create New
2) Restore Existing
Please choose: 1

You are creating an xud node key and underlying wallets. All will be secured by
a single password provided below.

Enter a password:
Re-enter password:

Error: 13 INTERNAL: could not initialize lnd-BTC: 14 UNAVAILABLE: failed to connect to all addresses
Do you want to create a new xud environment or restore an existing one?
1) Create New
2) Restore Existing
Please choose: 1

You are creating an xud node key and underlying wallets. All will be secured by
a single password provided below.

Enter a password:
Re-enter password:

Error: 14 UNAVAILABLE: lnd-LTC is Disconnected
Do you want to create a new xud environment or restore an existing one?
1) Create New
2) Restore Existing
Please choose: 1

You are creating an xud node key and underlying wallets. All will be secured by
a single password provided below.

...

To Reproduce Run master branch simnet

Expected behavior It should get over with the error and get successful "xucli create" at the very first time.

Additional context lndbtc.log lndltc.log xud.log

[kilrau]

Looks like macOS? How about in a linux VM?

[reliveyy]

Looks like macOS? How about in a linux VM?

It's all the same. If you type fast enough you will see this.

[kilrau]

I typed very fast and didn't see this. Did you manage to get into this? @raladev

If so, we have to add another condition for the create call to hang until lnd's become responsive.

[raladev]

I typed very fast and didn't see this. Did you manage to get into this? @raladev

If so, we have to add another condition for the create call to hang until lnd's become responsive.

After go downgrade i don't see this error. Usually if I typed very fast I see only 'xud still starting, wait please' msg.

But depends on previous errors i can say that u can get this error if your lnd node stuck on bootstrapping phase because of internet troubles or just starts too long because of performance troubles.

[kilrau]

As just discussed with @reliveyy , we are adding a lnd log check for this line:

2020-05-21 11:11:10.161 [INF] LTND: Waiting for wallet encryption password. Use `lncli create` to create a wallet, `lncli unlock` to unlock an existing wallet, or `lncli changepassword` to change the password of an existing wallet and unlock it.

for both lnd's in the Warming up... phase and remove the standard 5s sleep.

[kilrau]

As just discussed with @reliveyy , we are adding a lnd log check for this line:

2020-05-21 11:11:10.161 [INF] LTND: Waiting for wallet encryption password. Use `lncli create` to create a wallet, `lncli unlock` to unlock an existing wallet, or `lncli changepassword` to change the password of an existing wallet and unlock it.

for both lnd's in the Warming up... phase and remove the standard 5s sleep.

I am not really sure what you did here, but you removed the 5s sleep in this PR. Is this issue closed?

[reliveyy]

@kilrau We cannot close this issue yet. The fixes in PR #469 is not destined to solve this issue. If we want to resolve this one, then we need to figure out the exact checkpoints of lnd when xucli create has errors like:

• Error: 13 INTERNAL: could not initialize lnd-LTC/BTC: 14 UNAVAILABLE: failed to connect to all addresses
• Error: 14 UNAVAILABLE: lnd-LTC/BTC is Disconnected

From the experiences of PR #469 we know that the CLI response and log file are not reliable.

[kilrau]

Gotcha, thx. Let's leave it open then (with low prio) since this only happens on very restricted internet connections. No need to work on this for now.

[kilrau]

One more condition for the warm up check:

1. unlock attempt failed due to xud not being ready
2. unlock attempt two seconds later succeeded

Enter master xud password: 
Error: 2 UNKNOWN: /root/.xud/nodekey.dat does not contain a valid ECDSA private key

Enter master xud password: 
xud was unlocked succesfully