search

raspiblitz / raspiblitz

Get your own Bitcoin & Lightning Node running - on a RaspberryPi with a nice LCD
MIT License
2.45k stars 520 forks source link

Test: Install on VM (Debian QEMU host and Debian guest) #2772

Closed adevoss closed 2 years ago

adevoss commented 2 years ago

Steps

adevoss commented 2 years ago

These two errors are displayed when executing the build sdcard script:

Adding user admin to group sudo
Done.
%sudo ALL=(ALL) NOPASSWD:ALL
mv: cannot stat './raspiblitz.info': No such file or directory

and

Cloning into 'raspiblitz'...
remote: Enumerating objects: 34915, done.
remote: Counting objects: 100% (2772/2772), done.
remote: Compressing objects: 100% (940/940), done.
remote: Total 34915 (delta 2035), reused 2526 (delta 1832), pack-reused 32143
Receiving objects: 100% (34915/34915), 99.86 MiB | 2.84 MiB/s, done.
Resolving deltas: 100% (26091/26091), done.
cp: -r not specified; omitting directory '/home/admin/raspiblitz/home.admin/config.scripts'
cp: -r not specified; omitting directory '/home/admin/raspiblitz/home.admin/setup.scripts'
adevoss commented 2 years ago

When I execute the bootstrap script I get:

Resetting the InfoFile: /home/admin/raspiblitz.info
ls: cannot access '/boot/stop': No such file or directory
ls: cannot access '/home/admin/provision.flag': No such file or directory
# OK - file exists: /home/admin/raspiblitz/pictures/startlogo6.png
*** Checking Log Size ***
OK - logs are at 66 MB - within safety limit

and at the end

Failed to enable unit: Unit file bitcoind.service does not exist.
rm: cannot remove '/home/admin/provision.flag': No such file or directory
openoms commented 2 years ago

As suggested in https://github.com/rootzoll/raspiblitz/issues/1500#issuecomment-988243489 use the dev branch to build from where the mentioned problem are fixed already.

# download the build script
wget https://raw.githubusercontent.com/rootzoll/raspiblitz/dev/build_sdcard.sh
# run
sudo bash build_sdcard.sh false false rootzoll dev headless
adevoss commented 2 years ago

Ah. I think I used v1.7 of build_sdcard.sh Now the dev version is used. wget https://raw.githubusercontent.com/rootzoll/raspiblitz/dev/build_sdcard.sh sudo ./build_sdcard.sh false false rootzoll dev headless true false

ld tests/plugins/test_selfdisable_after_getmanifest
mrkd doc/lightning-cli.1.md
Traceback (most recent call last):
  File "/usr/local/bin/mrkd", line 5, in <module>
    from mrkd import main
  File "/usr/local/lib/python3.9/dist-packages/mrkd/__init__.py", line 23, in <module>
    class ReferenceLexer(mistune.InlineLexer):
AttributeError: module 'mistune' has no attribute 'InlineLexer'
make: *** [doc/Makefile:107: doc/lightning-cli.1] Error 1
rm external/x86_64-linux-gnu/libwally-core-build/src/secp256k1/libsecp256k1.la
cp tools/hsmtool tools/lightning-hsmtool
install cli/lightning-cli lightningd/lightningd tools/lightning-hsmtool /usr/local/bin
install lightningd/lightning_channeld lightningd/lightning_closingd lightningd/lightning_connectd lightningd/lightning_dualopend lightningd/lightning_gossipd lightningd/lightning_hsmd lightningd/lightning_onchaind lightningd/lightning_openingd lightningd/lightning_websocketd /usr/local/libexec/c-lightning
[ -z "plugins/autoclean plugins/bcli plugins/fetchinvoice plugins/funder plugins/topology plugins/keysend plugins/offers plugins/pay plugins/txprepare plugins/spenderp" ] || install plugins/autoclean plugins/bcli plugins/fetchinvoice plugins/funder plugins/topology plugins/keysend plugins/offers plugins/pay plugins/txprepare plugins/spenderp /usr/local/libexec/c-lightning/plugins
mrkd doc/lightning-cli.1.md
Traceback (most recent call last):
  File "/usr/local/bin/mrkd", line 5, in <module>
    from mrkd import main
  File "/usr/local/lib/python3.9/dist-packages/mrkd/__init__.py", line 23, in <module>
    class ReferenceLexer(mistune.InlineLexer):
AttributeError: module 'mistune' has no attribute 'InlineLexer'
make: *** [doc/Makefile:107: doc/lightning-cli.1] Error 1
openoms commented 2 years ago

fixed with https://github.com/rootzoll/raspiblitz/commit/2807623716944de66a1174762017118c700a7393 you can just run the updated script again

adevoss commented 2 years ago

fixed with 2807623 you can just run the updated script again

wget https://raw.githubusercontent.com/rootzoll/raspiblitz/dev/build_sdcard.sh sudo ./build_sdcard.sh false false rootzoll dev headless true false then ./_bootstrap.sh: line 300: HDD/SSD connected: ${$hddCandidate}: bad substitution removed the second '$'

/home/admin/_bootstrap.sh First time:

OK - logs are at 59 MB - within safety limit

# /home/admin/config.scripts/blitz.datadrive.sh status

Nothing happens (I think) Edit: Pressed Ctrl-C

Run a second time:

# Deleting the temp folder/drive (keeping SWAP file) ..
# OK cleaning done.
OK: subscription data directory exists
Failed to enable unit: Unit file bitcoind.service does not exist.
rm: cannot remove '/home/admin/provision.flag': No such file or directory

Returns to prompt

openoms commented 2 years ago

so once you built the OS switch off and attach another virtual disk see: https://github.com/rootzoll/raspiblitz/blob/665e718451d32a668c38979ad4289a9965fc2acd/alternative.platforms/README.md#virtual-machine

adevoss commented 2 years ago

That's what I did.

sudo /home/admin/_bootstrap.sh

The reason I login as another user is I then can see what happens. When I login as admin a blue screen is displayed with 'Please wait' (or similar)

openoms commented 2 years ago

Try: use one disk only to create the build

once ready switch off and attach second disk

log in with admin (psw: raspiblitz)

open a second terminal to monitor the logs (can exit to the terminal from admin as well with ctrl+c and return with the command: blitz)

adevoss commented 2 years ago

Done. Created a pr (https://github.com/rootzoll/raspiblitz/pull/2785)

*** ADDING MAIN USER admin ***
adduser: The group `admin' does not exist.
chpasswd: (user admin) pam_chauthtok() failed, error:
Authentication token manipulation error
chpasswd: (line 1, user admin) password not changed
adduser: The user `admin' does not exist.
chsh: user 'admin' does not exist
%sudo ALL=(ALL) NOPASSWD:ALL
./build_sdcard.sh: line 565: /home/admin/raspiblitz.info: No such file or directory
./build_sdcard.sh: line 566: /home/admin/raspiblitz.info: No such file or directory
./build_sdcard.sh: line 567: /home/admin/raspiblitz.info: No such file or directory
mv: cannot stat './raspiblitz.info': No such file or directory
chmod: cannot access '/home/admin/raspiblitz.info': No such file or directory

...

*** SHELL SCRIPTS & ASSETS ***
./build_sdcard.sh: line 597: cd: /home/admin/: No such file or directory
adevoss commented 2 years ago

build_sdcard.sh script: No errors (accept when trying to remove office which is not installed so that is expected)

Login with admin: Displays a menu with three(?) items. One of them is a fresh install. Don't want to loose my data so I shutdown and tried login under my user name, sudo ./_bootstrap.sh and got: Failed to enable unit: Unit file bitcoind.service does not exist.

openoms commented 2 years ago

not clear what you are trying to do.

Are you mounting a disk with data on it? use a new virtual image or one which has been used with the raspiblitz scripts already

adevoss commented 2 years ago

What I'm trying to do is install on a VM with two disks (OS/data) The datadisk has raspiblitz data.

Debian is installed with only one disk added to VM. The datadisk is added to VM (not mounted) when logging in.

build_sdcard.sh is OK. Login as 'admin' is not.

adevoss commented 2 years ago

Now looking into using /boot/stop and starting _provisioning_.sh manually.

openoms commented 2 years ago

What I would do is let raspiblitz set up on a new empty disk to have the format right.

Then:

adevoss commented 2 years ago

third disk

Unfortunately I don't have the disk space for this. So I did the following:

Provisioning finished but the blockchain isn't synced (didn't see it) and

sudo service lnd status
Unit lnd.service could not be found.

bitcoind is running

What I would expect is the syncing blockchain message (from 97 or so to 100% in my case)

openoms commented 2 years ago

Sorry once you go completely custom I can't reproduce the state and it is very difficult to help. Would be good to find a way to attach a used disk and have it recognised reliable by finding a general solution, but there are just so many edge-cases.

BTW the dev branch has just changed a lot with https://github.com/rootzoll/raspiblitz/pull/2790

adevoss commented 2 years ago

go completely custom

What do you mean by that? Is a Debian VM custom? Or dev branch? The manual execution of scripts is to find the cause or a workaround.

so many edge-cases

True. I don't mind testing.

find a way to attach a used disk

The disk is recognized by the install.

What I see is that my raspiblitz.conf file is/was missing values. For what reason I don't know. Haven't missed a release since v1.2 I think.

the dev branch has just changed a lot with #2790

Then start with a fresh Debian install again.

openoms commented 2 years ago

not sure what this script is:

sudo thunderbult.sh on

So you have disk image you have been running in a VM before? Or are you mounting the physical disk as an image for the VM (this is tricky in itself)?

adevoss commented 2 years ago

not sure what this script is:

This was a typo. I meant sudo /home/admin/config.scripts/bonus.thunderhub.sh on

So you have disk image you have been running in a VM before?

Yes. My setup is a VM with two virtual disks (os/raspiblitz data) being two files in raw format. No passthrough of physical disk to VM if that's what you meant.

adevoss commented 2 years ago

Steps for me to install

ssh session 1: sudo rm /home/admin/provision.flag sudo umount /mnt/hdd sudo echo "passwordA=secret" >> /var/cache/raspiblitz/temp/raspiblitz.setup sudo ./_bootstrap.sh (in loop) (Need to install the following packages: next) Ctrl+c sudo /home/admin/config.scripts/bonus.thunderhub.sh on sudo reboot sudo rm /home/admin/provision.flag sudo mkdir /var/cache/raspiblitz/temp sudo echo "passwordA=secret" >> /var/cache/raspiblitz/temp/raspiblitz.setup sudo service bitcoind stop sudo umount /mnt/hdd sudo /home/admin/_bootstrap.sh logout login as admin (Final reboot message displayed) Press to reboot login as admin Press to start Recovery

ssh session 2: (when in loop) sudo ./_cache.sh set state waitprovision tail -f /home/admin/raspiblitz.log (Need to install the following packages: next) Ctrl+c (after _bootstrap.sh is started for a 2nd time) tail -f /home/admin/raspiblitz.log

Had to reboot multiple times. Can't remember exactly how/what. Will try again from after pr's and issues (typos) are merged.

Now 'Blockchain Progress: waiting for peers' Not sure all is OK. It is taking too long it seems.

adevoss commented 2 years ago

Next fresh install starting with Debian installed. sudo ./build_sdcard.sh false false rootzoll dev headless true false OK.

Login as admin: Starting SSH user interface ... (please wait) setupPhase(boot) state() FAIL: /home/admin/raspiblitz.info does not exist or missing state. Check logs & bootstrap.service for errors and report to devs.

It was just too soon. After a minute (or less) logged in again. In the log there was: cat: /mnt/hdd/tor/debuglogs/hostname: No such file or directory

After the final reboot lnd is not running: [WRN] LTND: Error validating config: ValidateConfig: error normalizing RPC listen addrs: parse address 0.0.0.0:10009 failed: dial tcp 127.0.0.1:9050: connect: connection refused

adevoss commented 2 years ago

Did another test with dev branch.

ls -l /usr/local/bin/lnd
ls: cannot access '/usr/local/bin/lnd': No such file or directory
openoms commented 2 years ago

where is this error. Share the context please.

adevoss commented 2 years ago

After the install this file is missing (See #2813)

adevoss commented 2 years ago

A second login to view the log file does not give a shell prompt but the same as the first login (blue install screen) Workaround: login under my own name.

openoms commented 2 years ago

After the install this file is missing (See #2813)

any errors during the build_sdcard.sh?

openoms commented 2 years ago

A second login to view the log file does not give a shell prompt but the same as the first login (blue install screen) Workaround: login under my own name.

You can press CTRL+C and exit to the terminal in both the first or the second terminal. Return with the command: raspiblitz or blitz

adevoss commented 2 years ago

any errors during the build_sdcard.sh?

No errors that stops the script. There are messages about Tor not being reloaded (or restarted?).

openoms commented 2 years ago

You can test again, there has been a good amount of Tor related changes in dev

adevoss commented 2 years ago

Tested: sudo ./build_sdcard.sh --interaction true --fatpack true --github-user rootzoll --branch dev --display headless --tweak-boot-drive false --wifi-region off

Result: Manually before executing build script sudo apt install unzip curl Modify build_sdcard.sh if ${fatpack}; then (line 688,745,755)

Only part of error below (if you need all I can repeat the install)

Processing ./contrib/pyln-client
  Preparing metadata (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/bitcoin/download/clightning-v0.10.2/contrib/pyln-client/setup.py'"'"'; __file__='"'"'/home/bitcoin/download/clightning-v0.10.2/contrib/pyln-client/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-8pmh9q9k
       cwd: /home/bitcoin/download/clightning-v0.10.2/contrib/pyln-client/
  Complete output (27 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>

I didn't reboot to start recovery/provisioning because of this error.

openoms commented 2 years ago

Did that break the install? Is it the same as here (which could be ignored): https://github.com/rootzoll/raspiblitz/issues/2662#issue-1034847654

adevoss commented 2 years ago

Did that break the install?

The python error did not, the if statements do.

Is it the same as here (which could be ignored):

I think it is. So I will continue after executing the build script.

adevoss commented 2 years ago

Syncing now. Both Blockchain and Lightning (99.5%).

adevoss commented 2 years ago

Now at 100%. The 'Node is Syncing' screen is still being displayed. Shouldn't I see the main menu when the node is synced?

adevoss commented 2 years ago

It took longer than expected. Now the main menu is displayed.

adevoss commented 2 years ago

Not tested all components but now #2887 works as well.

openoms commented 2 years ago

just did a clean install on Debian with virsh Debian Bullseye VM. With the build command: sudo bash build_sdcard.sh -f true -b dev -d headless -t false -w off

Looking good so far.

openoms commented 2 years ago

This shoudl work now, see the notes at https://github.com/rootzoll/raspiblitz/tree/dev/alternative.platforms#virtual-machine