could not create keystore

kilrau commented 4 years ago

Background

Following the RaspiXUD guide Light Setup on a Pi3 B+, I get:

You are creating an xud node key and underlying wallets. All will be secured by
a single password provided below.

Enter a password: 
Re-enter password: 

Error: 13 INTERNAL: could not create keystore: Error: Command failed: ./seedutil/seedutil keystore -pass= -path=/root/.raiden abstract hair essence scrap future finger rifle innocent hill apology office river elite hurdle bomb chest multiply garment float alien practice abstract abc xyz

❌ Failed to launch mainnet environment. For more details, see /home/xud/.xud-docker/mainnet/mainnet.log

mainnet.log

Your environment

version of xud: xud-docker mainnet
which operating system (uname -a on *Nix): 19.10 arm64
version of btcd, bitcoind, ltcd, litecoind, geth, parity, raiden, lnd or other chain backend:
any other relevant environment details:

Steps to reproduce

Not clear, only managed to get this once.

kilrau commented 4 years ago

Test 2, deleted directories and docker images:

You are creating an xud node key and underlying wallets. All will be secured by
a single password provided below.

Enter a password: 
Re-enter password: 

❌ Failed to launch mainnet environment. For more details, see /home/xud/.xud-docker/mainnet/mainnet.log

sangaman commented 4 years ago

This has only happened once? What's strange is that abc and xyz are not part of the word list, that's most likely why the call failed in your first test, but I don't know why we would have gotten those words in the first place.

I'm not sure what went wrong with the second test, do you have the log for that one?

kilrau commented 4 years ago

abc and xyz

I edited the last two words, not really sure why though :)

This has only happened once?

So far yes. I am trying to reproduce.

kilrau commented 4 years ago

I tested again from scratch on the same Pi3 B+ with an officially recommended power cable to exclude this as reason for the behavior above.

Now the setup just got stuck (kept the shell open for 30 mins) at:

You are creating an xud node key and underlying wallets. All will be secured by
a single password provided below.

Enter a password: 
Re-enter password:

and the Pi just died. Here the last lines of mainnet.log, showing the Unexpected xucli create error:

2020-01-30 18:52:16.577 DEBUG 1 --- [MainThread] launcher.node.litecoind: Started container: mainnet_litecoind_1 (53317d3576c94596cc6ac28ccfdef895d07a272f71a33ffbbf25944535a10633)
2020-01-30 18:52:16.588 DEBUG 1 --- [MainThread] launcher.node.lndbtc: Started container: mainnet_lndbtc_1 (bb9e2b6de89fe9c3e4a0ed6b1e944761b7a4a7ce8a10b144290cc430074e0325)
2020-01-30 18:52:16.599 DEBUG 1 --- [MainThread] launcher.node.lndltc: Started container: mainnet_lndltc_1 (e3dbf52f4727205e837a5b7b5fb35752b494218995e64fd7b0fa35bdb2e63527)
2020-01-30 18:52:16.611 DEBUG 1 --- [MainThread] launcher.node.raiden: Started container: mainnet_raiden_1 (9778ac2f18392483014aa49bd5d38aecf6e5622e5c58db3736d42fc8a9719572)
2020-01-30 18:52:16.619 DEBUG 1 --- [MainThread] launcher.node.xud: Started container: mainnet_xud_1 (ebf5327c2ad0b659a591320fd5819e03a3ff7b830ac6e0f225b4fef2842e3435)
2020-01-30 18:52:21.378 DEBUG 1 --- [MainThread] launcher.node.lndbtc: lncli -n mainnet -c bitcoin getinfo (exit_code=1)
[lncli] open /root/.lnd/tls.cert: no such file or directory
2020-01-30 18:52:23.073 DEBUG 1 --- [MainThread] launcher.node.lndbtc: lncli -n mainnet -c bitcoin getinfo (exit_code=1)
[lncli] open /root/.lnd/tls.cert: no such file or directory
2020-01-30 18:52:24.595 DEBUG 1 --- [MainThread] launcher.node.lndbtc: lncli -n mainnet -c bitcoin getinfo (exit_code=1)
[lncli] open /root/.lnd/tls.cert: no such file or directory
2020-01-30 18:52:26.100 DEBUG 1 --- [MainThread] launcher.node.lndbtc: lncli -n mainnet -c bitcoin getinfo (exit_code=1)
[lncli] unable to read macaroon path (check the network setting!): open /root/.lnd/data/chain/bitcoin/mainnet/admin.macaroon: no such file or directory
2020-01-30 18:52:31.851 DEBUG 1 --- [MainThread] launcher.node.xud: xucli getinfo -j (exit_code=1)
/app/node_modules/yargs/yargs.js:1175
      else throw err
           ^

Error: ENOENT: no such file or directory, open '/root/.xud/tls.cert'
    at Object.openSync (fs.js:440:3)
    at Object.readFileSync (fs.js:342:35)
    at Object.exports.loadXudClient (/app/dist/cli/command.js:23:31)
    at Object.exports.handler (/app/dist/cli/commands/getinfo.js:69:15)
    at Object.runCommand (/app/node_modules/yargs/lib/command.js:237:40)
    at Object.parseArgs [as _parseArgs] (/app/node_modules/yargs/yargs.js:1090:30)
    at Object.get [as argv] (/app/node_modules/yargs/yargs.js:1024:21)
    at Object.<anonymous> (/app/bin/xucli:40:3)
    at Module._compile (internal/modules/cjs/loader.js:955:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '/root/.xud/tls.cert'
}
2020-01-30 18:52:39.973 DEBUG 1 --- [MainThread] launcher.node.xud: xucli getinfo -j (exit_code=1)
/app/node_modules/yargs/yargs.js:1175
      else throw err
           ^

Error: ENOENT: no such file or directory, open '/root/.xud/tls.cert'
    at Object.openSync (fs.js:440:3)
    at Object.readFileSync (fs.js:342:35)
    at Object.exports.loadXudClient (/app/dist/cli/command.js:23:31)
    at Object.exports.handler (/app/dist/cli/commands/getinfo.js:69:15)
    at Object.runCommand (/app/node_modules/yargs/lib/command.js:237:40)
    at Object.parseArgs [as _parseArgs] (/app/node_modules/yargs/yargs.js:1090:30)
    at Object.get [as argv] (/app/node_modules/yargs/yargs.js:1024:21)
    at Object.<anonymous> (/app/bin/xucli:40:3)
    at Module._compile (internal/modules/cjs/loader.js:955:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:991:10) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '/root/.xud/tls.cert'
}
2020-01-30 18:52:45.574 DEBUG 1 --- [MainThread] launcher.node.xud: xucli getinfo -j (exit_code=1)
xud is starting... try again in a few seconds
2020-01-30 18:52:50.709 DEBUG 1 --- [MainThread] launcher.node.xud: xucli getinfo -j (exit_code=1)
xud is locked, run 'xucli unlock' or 'xucli create' then try again
2020-01-30 18:52:50.710 DEBUG 1 --- [MainThread] launcher.node.xud: cli: create
2020-01-30 18:53:30.297 ERROR 1 --- [MainThread] launcher.Launcher: Failed to launch
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/launcher/launcher.py", line 651, in launch
    env.start()
  File "/usr/local/lib/python3.8/site-packages/launcher/launcher.py", line 631, in start
    self.check_wallets()
  File "/usr/local/lib/python3.8/site-packages/launcher/launcher.py", line 618, in check_wallets
    self.xucli_create_wrapper(xud)
  File "/usr/local/lib/python3.8/site-packages/launcher/launcher.py", line 577, in xucli_create_wrapper
    xud.cli("create", self._shell)
  File "/usr/local/lib/python3.8/site-packages/launcher/node/base.py", line 198, in cli
    raise exception
Exception: Unexpected xucli create error happens
2020-01-30 18:53:30.907 DEBUG 1 --- [MainThread] launcher.shell.Shell: stop
2020-01-30 18:53:30.909 DEBUG 1 --- [InputHandler] launcher.shell.InputHandler: ch='eof', self._cmd=<Command value= index=0 uncommitted=None>
2020-01-30 18:53:30.910 DEBUG 1 --- [InputHandler] launcher.shell.InputHandler: End
2020-01-30 18:53:31.890 DEBUG 1 --- [EventLoop] launcher.shell.EventLoop: End

Since it's working flawlessly on the Pi4, I suspect it has to do with the low-end CPU. Could you try the setup on a similarly spec'd ARM device than the Pi3 B+? @sangaman

kilrau commented 4 years ago

Also: how do you handle timeouts in https://github.com/ExchangeUnion/xud/pull/1376 ?

kilrau commented 4 years ago

Ran it again, watching the Pi 3's resources with htop on the side, nothing was in particular spiking when I created the node. This time it interstingly failed again with:

xud@ubuntu:~$ xud -b feat/arm
1) Simnet
2) Testnet
3) Mainnet
Please choose the network: 3
🚀 Launching mainnet environment
🌍 Checking for updates ...
All up-to-date.

You are creating an xud node key and underlying wallets. All will be secured by
a single password provided below.

Enter a password: 
Re-enter password: 

Error: 13 INTERNAL: could not create keystore: Error: Command failed: ./seedutil/seedutil keystore -pass= -path=/root/.raiden absent unhappy bonus patient outside comic pen select leopard clap hurt upper bargain flag canvas scissors cause gospel ski wasp sign race gym lens

❌ Failed to launch mainnet environment. For more details, see /home/xud/.xud-docker/mainnet/mainnet.log

Just starting the environment again immediately after this (no deleting of the mainnet data folders) resulted in this:

xud@ubuntu:~$ xud -b feat/arm
1) Simnet
2) Testnet
3) Mainnet
Please choose the network: 3
🚀 Launching mainnet environment
🌍 Checking for updates ...
All up-to-date.

You are creating an xud node key and underlying wallets. All will be secured by
a single password provided below.

Enter a password: 
Re-enter password: 

Error: 13 INTERNAL: could not generate aezeed
❌ Failed to launch mainnet environment. For more details, see /home/xud/.xud-docker/mainnet/mainnet.log

mainnet.log

engwarrior commented 4 years ago

Pls provide estimate for this task.

kilrau commented 4 years ago

@sangaman is already working on this @engwarrior

kilrau commented 4 years ago

Good news @sangaman , @michael1011 figured this out: this is because the seedutil crashed because the machine running out of RAM. Solution: Create a swap file on a 1GB RAM machine, update official recommendation to 2GB RAM. See https://github.com/ExchangeUnion/docs/issues/88

Closing here.

ExchangeUnion / xud