Closed sedamil closed 1 year ago
pablomendezroyo
commented
1 year ago Hey! looks like Teku is not been able to reach the Execution Client because the JWT is wrong. In the execution client, It's been used a different JWT for each consensus client. To start up the execution client with the right JWT you must select your client at http://my.dappnode/#/stakers/ethereum
sedamil
commented
1 year ago Thank you, I was trying to run two consensus clients thinking that they both needed to be synched for me to switch between them. So when prysm was running, teku was not able to work with the execution client, for teku to run, prysm has to be stopped. Now that I have one going at a time its not an issue.
pablomendezroyo
commented
1 year ago @sedamil cool! yeah right now not possible to run multiple at the same time, it will be possible in the future
pablomendezroyo
commented
1 year ago Not an issue
Teku syncs right after install, however, hangs up and is unable to sync shortly thereafter:
2022-12-04 17:44:45,531 main INFO Configuring logging for destination: console 2022-12-04 17:44:45,542 main INFO Logging includes events: true 2022-12-04 17:44:45,543 main INFO Logging includes validator duties: true 2022-12-04 17:44:45,543 main INFO Logging includes color: true 2022-12-04 17:44:45,562 main INFO Include P2P warnings set to: false 2022-12-04 17:44:46.116 INFO - Teku version: teku/v22.10.2/linux-x86_64/-eclipseadoptium-openjdk64bitservervm-java-17 2022-12-04 17:44:46.119 INFO - This software is licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 2022-12-04 17:44:46.332 INFO - JWT secret loaded from /jwtsecret 2022-12-04 17:44:46.335 INFO - Using execution engine at http://geth.dappnode:8551 2022-12-04 17:44:46.335 INFO - Enabling Builder Circuit Breaker 2022-12-04 17:44:46.361 INFO - Execution Engine version: KILNV2 2022-12-04 17:44:46.376 INFO - Eth1 endpoint not provided, using execution engine endpoint for eth1 data 2022-12-04 17:44:46.430 INFO - validator-api-enabled is false, not starting rest api. 2022-12-04 17:44:46.433 INFO - Storing beacon chain data in: /opt/teku/data/beacon 2022-12-04 17:44:46.435 INFO - Loading 0 validator keys... 2022-12-04 17:44:46.437 INFO - Loaded 0 Validators: 2022-12-04 17:44:46.461 INFO - Using optimized BLST library 2022-12-04 17:44:46.469 INFO - Starting metrics http service on 0.0.0.0:8008 2022-12-04 17:44:46.570 INFO - Metrics service started and listening on 0.0.0.0:8008 2022-12-04 17:44:46.573 INFO - Beacon data directory set to: /opt/teku/data/beacon 2022-12-04 17:44:46.645 INFO - Created leveldb2 Hot and Finalized database (leveldb2) at /opt/teku/data/beacon/db 2022-12-04 17:44:46.706 INFO - Using default implementation for ThreadExecutor 2022-12-04 17:44:46.716 INFO - Initialized Scheduler Signaller of type: class org.quartz.core.SchedulerSignalerImpl 2022-12-04 17:44:46.716 INFO - Quartz Scheduler v.2.3.2 created. 2022-12-04 17:44:46.717 INFO - RAMJobStore initialized. 2022-12-04 17:44:46.717 INFO - Scheduler meta-data: Quartz Scheduler (v2.3.2) 'QuartzScheduler' with instanceId 'NON_CLUSTERED' Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally. NOT STARTED. Currently in standby mode. Number of jobs executed: 0 Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 1 threads. Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.
2022-12-04 17:44:46.718 INFO - Quartz scheduler 'QuartzScheduler' initialized from an externally provided properties instance. 2022-12-04 17:44:46.718 INFO - Quartz scheduler version: 2.3.2 2022-12-04 17:44:46.733 INFO - Initializing storage 2022-12-04 17:44:46.735 INFO - Storage initialization complete 2022-12-04 17:44:46.736 INFO - Loading initial state from https://checkpoint-sync.dappnode.io/eth/v2/debug/beacon/states/finalized 2022-12-04 17:45:05.047 INFO - Loaded initial state at epoch 164976 (state root = 0xf431dabad469471a231e5fc71a5544f7659d4264c6ffe5bb1fff7cda77819ef3, block root = 0xc4e7c615d9efd8c0d83fcff44be75aa6c3d35f39351905640e35bbf81c72e8ad, block slot = 5279232). Please ensure that the supplied initial state corresponds to the latest finalized block as of the start of epoch 164976 (slot 5279232). 2022-12-04 17:45:05.731 INFO - Async storage updates enabled 2022-12-04 17:45:05.742 WARN - Rest API is enabled but no default fee recipient has been configured via the validators-proposer-default-fee-recipient option! It is strongly recommended to configure it to avoid possible block production failures in case the node has not been prepared for potential proposers by the Validator Client. 2022-12-04 17:45:06.050 ERROR - Execution Client request failed. Check the same JWT secret is configured for Teku and the Execution Client. org.web3j.protocol.exceptions.ClientConnectionException: Invalid response received: 403; signature is invalid
at org.web3j.protocol.http.HttpService.performIO(HttpService.java:178) ~[core-4.9.4.jar:?] at org.web3j.protocol.Service.send(Service.java:48) ~[core-4.9.4.jar:?] at org.web3j.protocol.Service.lambda$sendAsync$0(Service.java:60) ~[core-4.9.4.jar:?] at org.web3j.utils.Async.lambda$run$1(Async.java:38) ~[core-4.9.4.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?] at java.lang.Thread.run(Unknown Source) ~[?:?] 2022-12-04 17:45:06.055 ERROR - Failed to update fork choice java.util.concurrent.CompletionException: tech.pegasys.teku.ethereum.executionlayer.InvalidRemoteResponseException: Invalid remote response from the execution client: Invalid response received: 403; signature is invalid
at java.util.concurrent.CompletableFuture.encodeThrowable(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.completeThrowable(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$UniApply.tryFire(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.complete(Unknown Source) ~[?:?] at tech.pegasys.teku.infrastructure.async.SafeFuture.lambda$propagateResult$3(SafeFuture.java:148) ~[teku-infrastructure-async-22.10.2.jar:22.10.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.complete(Unknown Source) ~[?:?] at tech.pegasys.teku.infrastructure.async.SafeFuture.lambda$propagateResult$3(SafeFuture.java:148) ~[teku-infrastructure-async-22.10.2.jar:22.10.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.uniWhenCompleteStage(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.whenComplete(Unknown Source) ~[?:?] at tech.pegasys.teku.infrastructure.async.SafeFuture.whenComplete(SafeFuture.java:603) ~[teku-infrastructure-async-22.10.2.jar:22.10.2] at tech.pegasys.teku.infrastructure.async.SafeFuture.whenComplete(SafeFuture.java:33) ~[teku-infrastructure-async-22.10.2.jar:22.10.2] at tech.pegasys.teku.infrastructure.async.SafeFuture.propagateResult(SafeFuture.java:143) ~[teku-infrastructure-async-22.10.2.jar:22.10.2] at tech.pegasys.teku.infrastructure.async.SafeFuture.lambda$exceptionallyCompose$33(SafeFuture.java:422) ~[teku-infrastructure-async-22.10.2.jar:22.10.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(Unknown Source) ~[?:?] at tech.pegasys.teku.infrastructure.async.SafeFuture.lambda$propagateResult$3(SafeFuture.java:146) ~[teku-infrastructure-async-22.10.2.jar:22.10.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[?:?] at java.util.concurrent.CompletableFuture.completeExceptionally(Unknown Source) ~[?:?] at org.web3j.utils.Async.lambda$run$1(Async.java:40) ~[core-4.9.4.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?] at java.lang.Thread.run(Unknown Source) ~[?:?] Caused by: tech.pegasys.teku.ethereum.executionlayer.InvalidRemoteResponseException: Invalid remote response from the execution client: Invalid response received: 403; signature is invalid
at tech.pegasys.teku.ethereum.executionlayer.ExecutionLayerManagerImpl.unwrapResponseOrThrow(ExecutionLayerManagerImpl.java:518) ~[teku-ethereum-executionlayer-22.10.2.jar:22.10.2] at tech.pegasys.teku.ethereum.executionlayer.ExecutionLayerManagerImpl.unwrapExecutionClientResponseOrThrow(ExecutionLayerManagerImpl.java:504) ~[teku-ethereum-executionlayer-22.10.2.jar:22.10.2] ... 30 more 2022-12-04 17:45:06.205 INFO - Generated new p2p private key and storing in: generated-node-key 2022-12-04 17:45:06.370 INFO - PreGenesis Local ENR: enr:-Iu4QAX240WSskvw2uP73Ov4W_Jr6JApFPmi7rkvcTXQA3pOOTwtQO4vDriLVGQyWz75ssm7gZEgnmiY2nphX9Sl1QMChGV0aDKQ9aX9QgAAAAD__4JpZIJ2NIlzZWNwMjU2azGhA6Lqrxw0f1M3gGiqzappNwKRXhoSFZa5PcVyyJlpIX2 2022-12-04 17:45:06.377 INFO - Using multipeer sync 2022-12-04 17:45:06.514 INFO - Logging initialized @22971ms to org.eclipse.jetty.util.log.Slf4jLog 2022-12-04 17:45:09.983 INFO - Starting libp2p network... 2022-12-04 17:45:09.987 INFO - Listening for connections on: /ip4/172.33.0.14/tcp/9105/p2p/16Uiu2HAmPcwxFWcghGQgnWi6pWLnc3QKGaKJiemMFarnRe1wNApd 2022-12-04 17:45:09.990 INFO - Starting discovery server on UDP port 9105 2022-12-04 17:45:09.999 INFO - Monitor has started. Waiting BELLATRIX fork activation. Polling every PT14S 2022-12-04 17:45:10.000 INFO - UDP discovery client started 2022-12-04 17:45:10.002 INFO - Begin historical sync of blocks prior to slot 5279232 2022-12-04 17:45:10.003 INFO - Scheduler QuartzScheduler$_NON_CLUSTERED started. 2022-12-04 17:45:10.004 INFO - Starting Javalin ... 2022-12-04 17:45:10.013 INFO - You are running Javalin 4.6.6 (released October 2, 2022). 2022-12-04 17:45:10.016 INFO - Syncing *** Target slot: 5279323, Head slot: 5279232, Waiting for execution layer sync, Connected peers: 0 2022-12-04 17:45:10.034 INFO - Local ENR: enr:-Iu4QG35OElBxptqeDWzy-052zYIyD29M7grr76TT3UQHV53FbMHsussNYcnIJKjUIsa6MufS5QAe81FLSbUwTmt59ADhGV0aDKQSibFiwIAAAD__4JpZIJ2NIlzZWNwMjU2azGhA6Lqrxw0_f1M3gGiqzappNwKRXhoSFZa5PcVyyJlpIX2 2022-12-04 17:45:10.034 INFO - Starting eth2 gossip 2022-12-04 17:45:10.075 INFO - Static file handler added: StaticFileConfig(hostedPath=/webjars/swagger-ui/4.10.3, directory=/META-INF/resources/webjars/swagger-ui/4.10.3/, location=CLASSPATH, precompress=false, aliasCheck=null, headers={Cache-Control=max-age=0}, skipFileFunction=tech.pegasys.teku.infrastructure.restapi.SwaggerUIBuilder$$Lambda$1216/0x0000000800777770@2ed5bc12). File system location: 'jar:file:/opt/teku/lib/swagger-ui-4.10.3.jar!/META-INF/resources/webjars/swagger-ui/4.10.3/' 2022-12-04 17:45:10.076 INFO - Listening on http://0.0.0.0:3500/ 2022-12-04 17:45:10.076 INFO - Javalin started in 73ms \o/ 2022-12-04 17:45:10.086 ERROR - Failed to update eth1 chain head org.web3j.protocol.exceptions.ClientConnectionException: Invalid response received: 403; signature is invalid
at org.web3j.protocol.http.HttpService.performIO(HttpService.java:178) ~[core-4.9.4.jar:?] at org.web3j.protocol.Service.send(Service.java:48) ~[core-4.9.4.jar:?] at org.web3j.protocol.Service.lambda$sendAsync$0(Service.java:60) ~[core-4.9.4.jar:?] at org.web3j.utils.Async.lambda$run$1(Async.java:38) ~[core-4.9.4.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?] at java.lang.Thread.run(Unknown Source) ~[?:?] 2022-12-04 17:45:10.088 WARN - Endpoint http://geth.dappnode:8551 is INVALID | Invalid response received: 403; signature is invalid
2022-12-04 17:45:11.002 INFO - Syncing *** Target slot: 5279324, Head slot: 5279232, Waiting for execution layer sync, Connected peers: 0 2022-12-04 17:45:12.089 ERROR - Failed to update eth1 chain head org.web3j.protocol.exceptions.ClientConnectionException: Invalid response received: 403; signature is invalid
at org.web3j.protocol.http.HttpService.performIO(HttpService.java:178) ~[core-4.9.4.jar:?] at org.web3j.protocol.Service.send(Service.java:48) ~[core-4.9.4.jar:?] at org.web3j.protocol.Service.lambda$sendAsync$0(Service.java:60) ~[core-4.9.4.jar:?] at org.web3j.utils.Async.lambda$run$1(Async.java:38) ~[core-4.9.4.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?] at java.lang.Thread.run(Unknown Source) ~[?:?] 2022-12-04 17:45:14.092 ERROR - Failed to update eth1 chain head org.web3j.protocol.exceptions.ClientConnectionException: Invalid response received: 403; signature is invalid
at org.web3j.protocol.http.HttpService.performIO(HttpService.java:178) ~[core-4.9.4.jar:?] at org.web3j.protocol.Service.send(Service.java:48) ~[core-4.9.4.jar:?] at org.web3j.protocol.Service.lambda$sendAsync$0(Service.java:60) ~[core-4.9.4.jar:?] at org.web3j.utils.Async.lambda$run$1(Async.java:38) ~[core-4.9.4.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?] at java.lang.Thread.run(Unknown Source) ~[?:?] 2022-12-04 17:45:16.094 ERROR - Failed to update eth1 chain head org.web3j.protocol.exceptions.ClientConnectionException: Invalid response received: 403; signature is invalid
at org.web3j.protocol.http.HttpService.performIO(HttpService.java:178) ~[core-4.9.4.jar:?] at org.web3j.protocol.Service.send(Service.java:48) ~[core-4.9.4.jar:?] at org.web3j.protocol.Service.lambda$sendAsync$0(Service.java:60) ~[core-4.9.4.jar:?] at org.web3j.utils.Async.lambda$run$1(Async.java:38) ~[core-4.9.4.jar:?] at java.util.concurrent.CompletableFuture$AsyncRun.run(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?] at java.lang.Thread.run(Unknown Source) ~[?:?] 2022-12-04 17:45:18.097 ERROR - Failed to update eth1 chain head org.web3j.protocol.exceptions.ClientConnectionException: Invalid response received: 403; signature is invalid