search

Consensys / web3signer

Web3Signer is an open-source signing service capable of signing on multiple platforms (Ethereum1 and 2, Filecoin) using private keys stored in an external vault, or encrypted on a disk.
https://docs.web3signer.consensys.net/
Apache License 2.0
190 stars 75 forks source link

Issue with loading BLST Library #791

Closed riordant closed 1 year ago

riordant commented 1 year ago

Hi, I am unable to run the eth2 signing. I am using a rasperry pi.

here is the output:

bin/web3signer eth2 --keystores-path=$PATH --keystores-password-file=$FILE --slashing-protection-db-url="jdbc:postgresql://localhost/web3signer" --slashing-protection-db-username=postgres --slashing-protection-db-password=password --slashing-protection-pruning-enabled=true
2023-05-26 22:34:36.654+01:00 | main | INFO  | Web3SignerApp | Web3Signer has started with args eth2,--keystores-path=$PATH,--keystores-password-file=$FILE,--slashing-protection-db-url=jdbc:postgresql://localhost/web3signer,--slashing-protection-db-username=postgres,--slashing-protection-db-password=password,--slashing-protection-pruning-enabled=true
2023-05-26 22:34:36.919+01:00 | main | INFO  | Web3SignerApp | Version = web3signer/v23.3.1/linux-arm_32/-raspbian-openjdkservervm-java-11
2023-05-26 22:34:42.196+01:00 | main | INFO  | Eth2SubCommand | Network: mainnet
Spec Name: PHASE0, Fork Epoch: 0, First Slot: 0
Spec Name: ALTAIR, Fork Epoch: 74240, First Slot: 2375680
Spec Name: BELLATRIX, Fork Epoch: 144896, First Slot: 4636672
Spec Name: CAPELLA, Fork Epoch: 194048, First Slot: 6209536

2023-05-26 22:34:42.340+01:00 | main | INFO  | HikariDataSource | HikariPool-1 - Starting...
2023-05-26 22:34:43.064+01:00 | main | INFO  | HikariPool | HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection@1c713c9
2023-05-26 22:34:43.082+01:00 | main | INFO  | HikariDataSource | HikariPool-1 - Start completed.
2023-05-26 22:34:44.285+01:00 | main | INFO  | HikariDataSource | HikariPool-2 - Starting...
2023-05-26 22:34:44.352+01:00 | main | INFO  | HikariPool | HikariPool-2 - Added connection org.postgresql.jdbc.PgConnection@90b1a6
2023-05-26 22:34:44.354+01:00 | main | INFO  | HikariDataSource | HikariPool-2 - Start completed.
2023-05-26 22:34:45.710+01:00 | pool-2-thread-1 | INFO  | SignerLoader | Loading signer configuration metadata files from .
2023-05-26 22:34:45.746+01:00 | pool-2-thread-1 | INFO  | SignerLoader | Signer configuration metadata files read in memory 0 in 00:00:00.024
2023-05-26 22:34:45.756+01:00 | ForkJoinPool-1-worker-3 | INFO  | SignerLoader | Parsing configuration metadata files
2023-05-26 22:34:45.762+01:00 | ForkJoinPool-1-worker-3 | INFO  | SignerLoader | Total configuration metadata files processed: 0
2023-05-26 22:34:45.763+01:00 | ForkJoinPool-1-worker-3 | INFO  | SignerLoader | Total signers loaded from configuration files: 0 in 00:00:00.005 with error count 0
2023-05-26 22:34:45.787+01:00 | pool-2-thread-1 | INFO  | Eth2Runner | Bulk loading keys from local keystores ... 
2023-05-26 22:34:53.282+01:00 | pool-2-thread-1 | INFO  | teku-status-log | Using portable BLST library.
2023-05-26 22:34:53.331+01:00 | pool-2-thread-1 | ERROR | BlstLoader | Couldn't load native BLS library
java.lang.ExceptionInInitializerError: null

thanks

siladu commented 1 year ago

Hi @riordant - which raspberry pi are you using exactly?

Have you got more logs that show more of the error?

riordant commented 1 year ago

Hi, It's a Raspberry Pi 4 Model B, running Raspberry Pi OS. These are the only logs that are relevant. It is ARM based. Is that an issue?

jframe commented 1 year ago

It could be an issue. The native library we use for BLS might not support that ARM architecture. If we don't support architecture, we should fall back to Java, but that might not be happening. It will be slower if it doesn't use the native library.

In any case, we will test this out on a Raspberry Pi 4.

riordant commented 1 year ago

Awesome, thanks.

usmansaleem commented 1 year ago

Tested against Raspberry Pi 3 Model B v1.2 with 64 bit Raspberry PI OS

uname -a
Linux raspberrypi 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr  3 17:24:16 BST 2023 aarch64 GNU/Linux

Web3Signer is able to load native BLST library successfully:

usman@raspberrypi:~/web3signer-23.3.1/bin $ ./web3signer --config-file /home/usman/config/config.yaml eth2
2023-06-08 12:02:23.363+10:00 | main | INFO  | Web3SignerApp | Web3Signer has started with args --config-file,/home/usman/config/config.yaml,eth2
2023-06-08 12:02:23.680+10:00 | main | INFO  | Web3SignerApp | Version = web3signer/v23.3.1/linux-aarch_64/-eclipseadoptium-openjdk64bitservervm-java-11
2023-06-08 12:02:32.392+10:00 | main | INFO  | Eth2SubCommand | Network: mainnet
Spec Name: PHASE0, Fork Epoch: 0, First Slot: 0
Spec Name: ALTAIR, Fork Epoch: 74240, First Slot: 2375680
Spec Name: BELLATRIX, Fork Epoch: 144896, First Slot: 4636672
Spec Name: CAPELLA, Fork Epoch: 194048, First Slot: 6209536

2023-06-08 12:02:34.429+10:00 | pool-2-thread-1 | INFO  | SignerLoader | Loading signer configuration metadata files from /home/usman/config/keys
2023-06-08 12:02:34.487+10:00 | pool-2-thread-1 | INFO  | SignerLoader | Signer configuration metadata files read in memory 1 in 00:00:00.047
2023-06-08 12:02:34.513+10:00 | ForkJoinPool-1-worker-3 | INFO  | SignerLoader | Parsing configuration metadata files
2023-06-08 12:02:35.130+10:00 | ForkJoinPool-1-worker-3 | INFO  | teku-status-log | Using portable BLST library.
2023-06-08 12:02:35.238+10:00 | ForkJoinPool-1-worker-3 | INFO  | BLS | BLS: loaded BLST library
2023-06-08 12:02:35.277+10:00 | ForkJoinPool-1-worker-3 | INFO  | SignerLoader | Total configuration metadata files processed: 1
2023-06-08 12:02:35.279+10:00 | ForkJoinPool-1-worker-3 | INFO  | SignerLoader | Total signers loaded from configuration files: 1 in 00:00:00.760 with error count 0
2023-06-08 12:02:35.315+10:00 | pool-2-thread-1 | INFO  | DefaultArtifactSignerProvider | Total signers (keys) currently loaded in memory: 1
2023-06-08 12:02:41.804+10:00 | main | INFO  | Runner | Web3Signer has started with TLS disabled, and ready to handle signing requests on 0.0.0.0:9000
usmansaleem commented 1 year ago

Tested against Raspberry Pi 3 Model B v1.2 with 32 bit Raspberry PI OS

The native BLST library (provided via Teku libraries) fails to load native implementation and raises an exception instead of reverting back to non-native implementation. This results in failure to load the private key. However, web3signer still starts.

./web3signer --config-file /home/usman/config/config.yaml eth2
2023-06-08 12:49:55.709+10:00 | main | INFO  | Web3SignerApp | Web3Signer has started with args --config-file,/home/usman/config/config.yaml,eth2
2023-06-08 12:49:55.865+10:00 | main | INFO  | Web3SignerApp | Version = web3signer/v23.3.1/linux-arm_32/zulu-java-11
2023-06-08 12:50:02.374+10:00 | main | INFO  | Eth2SubCommand | Network: mainnet
Spec Name: PHASE0, Fork Epoch: 0, First Slot: 0
Spec Name: ALTAIR, Fork Epoch: 74240, First Slot: 2375680
Spec Name: BELLATRIX, Fork Epoch: 144896, First Slot: 4636672
Spec Name: CAPELLA, Fork Epoch: 194048, First Slot: 6209536

2023-06-08 12:50:03.909+10:00 | pool-2-thread-1 | INFO  | SignerLoader | Loading signer configuration metadata files from /home/usman/config/keys
2023-06-08 12:50:03.946+10:00 | pool-2-thread-1 | INFO  | SignerLoader | Signer configuration metadata files read in memory 1 in 00:00:00.027
2023-06-08 12:50:03.961+10:00 | ForkJoinPool-1-worker-3 | INFO  | SignerLoader | Parsing configuration metadata files
2023-06-08 12:50:04.451+10:00 | ForkJoinPool-1-worker-3 | INFO  | teku-status-log | Using portable BLST library.
2023-06-08 12:50:04.484+10:00 | ForkJoinPool-1-worker-3 | ERROR | BlstLoader | Couldn't load native BLS library
java.lang.ExceptionInInitializerError: null
    at java.lang.Class.forName0(Native Method) ~[?:?]
    at java.lang.Class.forName(Class.java:315) ~[?:?]
    at tech.pegasys.teku.bls.impl.blst.BlstLoader.loadBlst(BlstLoader.java:49) ~[bls-23.3.1.jar:23.3.1]
    at tech.pegasys.teku.bls.impl.blst.BlstLoader.<clinit>(BlstLoader.java:38) ~[bls-23.3.1.jar:23.3.1]
    at tech.pegasys.teku.bls.BLS.resetBlsImplementation(BLS.java:56) ~[bls-23.3.1.jar:23.3.1]
    at tech.pegasys.teku.bls.BLS.<clinit>(BLS.java:48) ~[bls-23.3.1.jar:23.3.1]
    at tech.pegasys.teku.bls.BLSSecretKey.fromBytes(BLSSecretKey.java:38) ~[bls-23.3.1.jar:23.3.1]
    at tech.pegasys.web3signer.signing.config.metadata.BlsArtifactSignerFactory.create(BlsArtifactSignerFactory.java:71) ~[web3signer-signing-23.3.1.jar:23.3.1]
    at tech.pegasys.web3signer.signing.config.metadata.FileRawSigningMetadata.createSigner(FileRawSigningMetadata.java:34) ~[web3signer-signing-23.3.1.jar:23.3.1]
Caused by: java.lang.RuntimeException: no blst in java.library.path: [/usr/java/packages/lib, /lib, /usr/lib]
    at supranational.blst.blstJNI.<clinit>(blstJNI.java:31) ~[jblst-0.3.8.jar:?]
    ... 49 more
Caused by: java.lang.ExceptionInInitializerError
    at tech.pegasys.teku.bls.BLSSecretKey.fromBytes(BLSSecretKey.java:38) ~[bls-23.3.1.jar:23.3.1]
    at tech.pegasys.web3signer.signing.config.metadata.BlsArtifactSignerFactory.create(BlsArtifactSignerFactory.java:71) ~[web3signer-signing-23.3.1.jar:23.3.1]
    at tech.pegasys.web3signer.signing.config.metadata.FileRawSigningMetadata.createSigner(FileRawSigningMetadata.java:34) ~[web3signer-signing-23.3.1.jar:23.3.1]
2023-06-08 12:50:04.576+10:00 | pool-2-thread-1 | WARN  | Eth2Runner | No BLS keys loaded. Check that the key store has BLS key config files
2023-06-08 12:50:04.583+10:00 | pool-2-thread-1 | INFO  | DefaultArtifactSignerProvider | Total signers (keys) currently loaded in memory: 0
2023-06-08 12:50:08.694+10:00 | main | INFO  | Runner | Web3Signer has started with TLS disabled, and ready to handle signing requests on 0.0.0.0:9000
usmansaleem commented 1 year ago

Web3Signer indirectly use https://github.com/ConsenSys/jblst which is currently not supported for 32 bit OS, which makes Web3Signer not supported on 32 bit OS.

Suggested workaround, use 64 bit OS on Raspberry PI.

usmansaleem commented 1 year ago

@riordant kindly have a look at above analysis. You may have to use 64 bit OS on your Raspberry PI instance.

usmansaleem commented 1 year ago

@riordant if you want to use Raspberry OS 32 bit, you have to rebuilt jblst jar on your Raspberry PI and replace the bundled jblst jar with it. Here are the instructions that I've followed: https://gist.github.com/usmansaleem/650813a81c41298e6b6f45137ef133cd

usmansaleem commented 1 year ago

Closing the issue, please reopen if you have further queries.