rhmaxdotorg / pulsechain-validator

PulseChain Validator Automated Setup Scripts
https://launchpad.pulsechain.com
38 stars 13 forks source link

Error Updating Client #38

Closed Sipheren closed 2 months ago

Sipheren commented 2 months ago

Hi,

I tried to do an update today and got the following error on both my nodes, is there anything I can do fix this?

Thanks

Compiling rustc-hash v1.1.0 Compiling time v0.3.34 Compiling p256 v0.13.2 error[E0282]: type annotations needed for Box<_> --> /home/node/.cargo/registry/src/index.crates.io-6f17d22bba15001f/time-0.3.34/src/format_description/parse/mod.rs:83:9 83 let items = format_items ^^^^^ ... 86 Ok(items.into()) ---- type must be known at this point

help: consider giving items an explicit type, where the placeholders _ are specified | 83 | let items: Box<_> = format_items | ++++++++

Compiling data-encoding-macro v0.1.14 Compiling multihash v0.19.1 Compiling hkdf v0.12.4 Compiling libsqlite3-sys v0.25.2 For more information about this error, try rustc --explain E0282. error: could not compile time (lib) due to 1 previous error warning: build failed, waiting for other jobs to finish... error: failed to compile lighthouse v2.5.0 (/opt/lighthouse/lighthouse), intermediate artifacts can be found at /opt/lighthouse/target. To reuse those artifacts with a future compilation, set the environment variable CARGO_TARGET_DIR to that path. make: *** [Makefile:51: install] Error 101

Step 3: Restarting PulseChain clients

Process is complete

rhmaxdotorg commented 2 months ago

Hmm looks like an error building lighthouse.

I don't remember seeing this before and last upgrade for clients I did a few months ago went OK.

I would try running the update script again (if you haven't already) or rebooting and trying the update script again (in case its some weird OS issue). Or you're still stuck, might need to reset the validator (by default it keeps blockchain data, so minimal downtime) using the reset script and then running the setup script again for the process.

eschatar commented 2 months ago

I was having the same issue when I upgraded my validator node. I decided to rebuild using the rhmax scripts since he doesn't use docker, which is how I built mine.

Anyway, for the CARGO_TARGET_DIR thing just add a line to the script right after it drops into the $LIGHTHOUSE_DIR and before it starts the make command to compile lighthouse that does a cargo update.

Example image attached: image

FYI, I also ended up having to downgrade the version of go that gets installed because I was getting a different error compiling geth due to an incompatibility with the newest version of go. Just modify the existing line

from: sudo snap install --classic go to: sudo snap install --channel=1.22/stable --classic go

Cheers!

rhmaxdotorg commented 2 months ago

Thank you @Sipheren! That's very helpful.

After digging into it further, it does look like Go versioning and Rust updates apparently broke some stuff. Just modified the update script (as I'm avoiding making changes to main setup script for now, unless anyone wants to volunteer to do some testing and send PR :) so it should make things smoother.

https://github.com/rhmaxdotorg/pulsechain-validator/commit/8988e68d4da8d85b479c77667a81ddf3ccab7eb1

Update available to be pulled from the repo directory and to grab the latest update-client.sh script.

Sipheren commented 2 months ago

Thank you @Sipheren! That's very helpful.

After digging into it further, it does look like Go versioning and Rust updates apparently broke some stuff. Just modified the update script (as I'm avoiding making changes to main setup script for now, unless anyone wants to volunteer to do some testing and send PR :) so it should make things smoother.

8988e68

Update available to be pulled from the repo directory and to grab the latest update-client.sh script.

Thank you for looking into this for me, I am running the new update script now, will let you know how it goes.

Cheers

Sipheren commented 2 months ago

EDIT: Well, one worked without an issue and one got the below error, I might just need to reset one of them.

Hi,

I pulled the new update script, it almost got to the end but I still got an error re the CARGO_TARGET_DIR, Which script do I need to edit to fix this, I saw you mention it above but I wasnt able to find that in the update script.

Or do you mean in the reset script, and to reset after making those changes?

error: Compiling boot_node v2.5.0 (/opt/lighthouse/boot_node) error: could not compile lighthouse (bin "lighthouse")

Caused by: process didn't exit successfully: /home/david/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc --crate-name lighthouse --edition=2021 lighthouse/src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=100 --crate-type bin --emit=dep-info,link -C opt-level=3 -C embed-bitcode=no --cfg 'feature="default"' --cfg 'feature="jemalloc"' --cfg 'feature="slasher-lmdb"' --check-cfg 'cfg(docsrs)' --check-cfg 'cfg(feature, values("default", "gnosis", "jemalloc", "modern", "portable", "slasher-lmdb", "slasher-mdbx", "spec-minimal", "write_ssz_files"))' -C metadata=ba1bb0e9a152234a -C extra-filename=-ba1bb0e9a152234a --out-dir /opt/lighthouse/target/release/deps -C strip=debuginfo -L dependency=/opt/lighthouse/target/release/deps --extern account_manager=/opt/lighthouse/target/release/deps/libaccount_manager-25bfb052d8fa4b31.rlib --extern account_utils=/opt/lighthouse/target/release/deps/libaccount_utils-8fd0172cf84dc463.rlib --extern beacon_node=/opt/lighthouse/target/release/deps/libbeacon_node-df90e8c463a6a8d8.rlib --extern bls=/opt/lighthouse/target/release/deps/libbls-83dfe7e558d5970b.rlib --extern boot_node=/opt/lighthouse/target/release/deps/libboot_node-32a149196abcce14.rlib --extern clap=/opt/lighthouse/target/release/deps/libclap-8b73c3ff02b2061a.rlib --extern clap_utils=/opt/lighthouse/target/release/deps/libclap_utils-942d73f87f01e358.rlib --extern database_manager=/opt/lighthouse/target/release/deps/libdatabase_manager-b535be276cdbcc42.rlib --extern directory=/opt/lighthouse/target/release/deps/libdirectory-317b262ee0f0ec4f.rlib --extern environment=/opt/lighthouse/target/release/deps/libenvironment-1d86ae2106e0d26f.rlib --extern eth2_network_config=/opt/lighthouse/target/release/deps/libeth2_network_config-2255feedf906f99e.rlib --extern ethereum_hashing=/opt/lighthouse/target/release/deps/libethereum_hashing-c7bb52a098548b00.rlib --extern futures=/opt/lighthouse/target/release/deps/libfutures-40ab1870252d4a63.rlib --extern lazy_static=/opt/lighthouse/target/release/deps/liblazy_static-3a90058cb3352bf2.rlib --extern lighthouse_metrics=/opt/lighthouse/target/release/deps/liblighthouse_metrics-3a544ed10ff73bcb.rlib --extern lighthouse_version=/opt/lighthouse/target/release/deps/liblighthouse_version-81405fd3b621d6a7.rlib --extern logging=/opt/lighthouse/target/release/deps/liblogging-1155fa73fd28ad6c.rlib --extern malloc_utils=/opt/lighthouse/target/release/deps/libmalloc_utils-ca2575a38d25deeb.rlib --extern serde=/opt/lighthouse/target/release/deps/libserde-1421578b3a757777.rlib --extern serde_json=/opt/lighthouse/target/release/deps/libserde_json-5586da699e736fcb.rlib --extern serde_yaml=/opt/lighthouse/target/release/deps/libserde_yaml-34fa446e4f12d168.rlib --extern slasher=/opt/lighthouse/target/release/deps/libslasher-8447d680ba29ea40.rlib --extern slog=/opt/lighthouse/target/release/deps/libslog-adc0d41668c735c5.rlib --extern sloggers=/opt/lighthouse/target/release/deps/libsloggers-ac6923bf6c103ead.rlib --extern task_executor=/opt/lighthouse/target/release/deps/libtask_executor-2dd6ec39bc6951d3.rlib --extern tracing_subscriber=/opt/lighthouse/target/release/deps/libtracing_subscriber-018f04835a1c20e7.rlib --extern types=/opt/lighthouse/target/release/deps/libtypes-f44a3a8f11cd1910.rlib --extern unused_port=/opt/lighthouse/target/release/deps/libunused_port-24ac1dda9982ad33.rlib --extern validator_client=/opt/lighthouse/target/release/deps/libvalidator_client-8185ed2fae972ea5.rlib --extern validator_manager=/opt/lighthouse/target/release/deps/libvalidator_manager-19fb69d120a0421e.rlib -L native=/opt/lighthouse/target/release/build/ring-5598e026b1ad1389/out -L native=/opt/lighthouse/target/release/build/ring-b4612ac34e6cc896/out -L native=/opt/lighthouse/target/release/build/blst-3fbc9c374512adaa/out -L native=/opt/lighthouse/target/release/build/c-kzg-f1e244b9a59a7f74/out -L native=/opt/lighthouse/target/release/build/libsqlite3-sys-c168531de6d5a089/out -L native=/opt/lighthouse/target/release/build/openssl-sys-f9f0b981e7e28049/out/openssl-build/install/lib -L native=/opt/lighthouse/target/release/build/leveldb-sys-225b1290842371c5/out/lib -L native=/opt/lighthouse/target/release/build/leveldb-sys-225b1290842371c5/out/lib -L native=/opt/lighthouse/target/release/build/lmdb-rkv-sys-2dc00fe3eb68e331/out -L native=/opt/lighthouse/target/release/build/jemalloc-sys-04bd3f2197c4dd40/out/build/lib (signal: 9, SIGKILL: kill) error: failed to compile lighthouse v2.5.0 (/opt/lighthouse/lighthouse), intermediate artifacts can be found at /opt/lighthouse/target. To reuse those artifacts with a future compilation, set the environment variable CARGO_TARGET_DIR to that path. make: *** [Makefile:51: install] Error 101

Step 3: Restarting PulseChain clients

eschatar commented 2 months ago

The screenshot I posted is where I made the change - in the pulsechain-validator-setup.sh script. The other change I made was in the same script, but that one fixed an issue I was having compiling geth, not lighthouse. Hope that clears it up.

Sipheren commented 2 months ago

The screenshot I posted is where I made the change - in the pulsechain-validator-setup.sh script. The other change I made was in the same script, but that one fixed an issue I was having compiling geth, not lighthouse. Hope that clears it up.

Hi, yeah, I worked it out :)

I am going to update the setup script and re-do the one that didnt work, thanks for providing this and helping :)

Cheers

Sipheren commented 2 months ago

Hi Mate,

Quick question, after the reset script and running the setup script again, I just have to wait for it sync and thats it right, I dont do the keys and all that again like I did in the initial setup?

Cheers

rhmaxdotorg commented 2 months ago

If I remember correctly (been a while since I had to do it), the reset script removes all the code and packages and only leaves blockchain data by default, which would mean you'd to import the keys again.

Sipheren commented 2 months ago

Hi,

Thanks for the reply. Its a little odd, I have 2 nodes.

One I reset, got the cargo update done then ran the setup script, waited a few hours and its all back working, no key import. I assume its working as it shows as active on the pls validator site.

The other one doesnt seem to be right, it seems to be running but the validator site still shows as down, would you know of a simple way to see what the issue might be?

I get this from the beacon:

david@pls01:~$ journalctl -u lighthouse-beacon.service -f Sep 15 08:20:26 pls01.skynet.local lh[59223]: Sep 14 22:20:26.740 INFO New RPC block received hash: 0xc249…0e48, slot: 4256686 Sep 15 08:20:30 pls01.skynet.local lh[59223]: Sep 14 22:20:30.001 INFO Synced slot: 4256687, block: 0xa7c8…960b, epoch: 133021, finalized_epoch: 133019, finalized_root: #, exec_hash: # (verified), peers: 100, service: slot_notifier Sep 15 08:20:35 pls01.skynet.local lh[59223]: Sep 14 22:20:35.366 INFO New block received root: ##, slot: 4256688 Sep 15 08:20:40 pls01.skynet.local lh[59223]: Sep 14 22:20:40.001 INFO Synced slot: 4256688, block: 0x2c22…fa10, epoch: 133021, finalized_epoch: 133019, finalized_root: 0xbb68…1afa, exec_hash: ## (verified), peers: 101, service: slot_notifier Sep 15 08:20:45 pls01.skynet.local lh[59223]: Sep 14 22:20:45.319 INFO New block received root: ##, slot: 4256689 Sep 15 08:20:50 pls01.skynet.local lh[59223]: Sep 14 22:20:50.001 INFO Downloading historical blocks est_time: 20 hrs 30 mins, speed: 9.60 slots/sec, distance: 708576 slots (11 weeks 5 days), service: slot_notifier

It may just need more time to sort it self out.....

Cheers

Sipheren commented 2 months ago

Edit to the above. All working, needed to reimport the keys for the one that was being odd. The other was fine though.

Cheers

rhmaxdotorg commented 2 months ago

Great, glad you got it working!