status-im / infra-nim-waku

Infrastructure for Nim Waku

https://github.com/status-im/nim-waku

4 stars 5 forks source link

Connecting waku v2 test fleets to an Ethereum node on Goerli testnet #59

Closed staheri14 closed 2 years ago

staheri14 commented 2 years ago

Problem

As part of waku-rln-relay protocol, waku v2 nodes are supposed to listen to some events emitted from a smart contract deployed on Goerli testnet and perform some operations internally. The necessary code for this is already implemented that is right now waku v2 nides can accept the address of a node on Goerli testnet via a dedicated config parameter i.e., eth-client-address which is the websocket address of a node (e.g., Infura). Once passed, the waku node makes the necessary connections for event subscription and further communication.

The current issue is to explore how it should be done on waku test fleets. My understanding is that 1) we need to have the address of an Ethereum node on Goerli testnet to which waku test fleets can connect 2) the corresponding config option eth-client-address should be added to https://github.com/status-im/infra-nim-waku/blob/master/ansible/group_vars/wakuv2-test.yml#L45 similar to other rln-relay parameters.

jakubgs commented 2 years ago

Does this have to be a dedicated Geth node on Goerli testnet, or would it be fine to connect to one of the Nimbus ones?

jakubgs commented 2 years ago

After discussing this with @staheri14 it appears it should be fine to use some of the Geth nodes we've deployed for Nimbus.

This will make this much simpler, and also cheaper.

jakubgs commented 2 years ago

I see no space for a JWT secret, so I assume there's only support for plain RPC:

admin@node-01.do-ams3.wakuv2.test:~ % d run --rm -it statusteam/nim-waku:deploy-v2-test --help | grep eth-
 --eth-account-address     Account address for the Ethereum testnet Goerli.
 --eth-account-privatekey  Account private key for the Ethereum testnet
 --eth-client-address      Ethereum testnet client address e.g.,
 --eth-mem-contract-address  Address of membership contract on an Ethereum

staheri14 commented 2 years ago

I see no space for a JWT secret, so I assume there's only support for plain RPC:

Ethereum clients implement a suite of JSON-RPC methods. Is the question about this? Or are there other concerns?

jakubgs commented 2 years ago

The main way CL clients communicate with EL clients is using Auth RPC, not plain RPC. That's what I'm referring to.

jakubgs commented 2 years ago

Change has been deployed:

https://github.com/status-im/infra-nimbus/commit/414923bf - nimbus.prater: open Geth RPC ports for wakuv2 fleets
https://github.com/status-im/infra-nim-waku/commit/5a47a72a - wakuv2-test: use nimbus.prater node for eth client

Result:

 > a all -a 'grep eth-client /docker/nim-waku-v2/docker-compose.yml'  
node-01.do-ams3.wakuv2.test | CHANGED | rc=0 >>
      --eth-client-address=http://linux-01.he-eu-hel1.nimbus.prater.wg:8546
node-01.gc-us-central1-a.wakuv2.test | CHANGED | rc=0 >>
      --eth-client-address=http://linux-01.he-eu-hel1.nimbus.prater.wg:8546
node-01.ac-cn-hongkong-c.wakuv2.test | CHANGED | rc=0 >>
      --eth-client-address=http://linux-01.he-eu-hel1.nimbus.prater.wg:8546

I'm using just the first Geth node from linux-01.he-eu-hel1.nimbus.prater.wg but if we want to in thee future we could have each node using it's own dedicated Geth node.

jakubgs commented 2 years ago

Some more not exactly related fixes:

https://github.com/status-im/infra-role-nim-waku/commit/57c296a4 - consul: re-enable websocket healthcheck
https://github.com/status-im/infra-role-nim-waku/commit/edfeed3b - consul: use 127.0.0.1 instead of localhost
https://github.com/status-im/infra-role-nim-waku/commit/d892e19d - docker: fix eth client address flag check

staheri14 commented 2 years ago

Thanks a lot @jakubgs for taking care of this!

The client addresses are HTTP, can we please pass a WebSocket address instead?

jakubgs commented 2 years ago

Okay. Don't you think the flag should indicate in the help message that it expects a websocket?

jakubgs commented 2 years ago

Fixed:

https://github.com/status-im/infra-nimbus/commit/fa3bb886 - nimbus.prater: open websocket port for wakuv2.test
https://github.com/status-im/infra-nimbus/commit/1f54ea1f - nimbus.prater: open websocket port to other hosts
https://github.com/status-im/infra-nim-waku/commit/725ff4a7 - wakuv2-test: use websocket port for eth client

 > a all -a 'grep eth-client /docker/nim-waku-v2/docker-compose.yml' 
node-01.do-ams3.wakuv2.test | CHANGED | rc=0 >>
      --eth-client-address=ws://linux-01.he-eu-hel1.nimbus.prater.wg:9547
node-01.gc-us-central1-a.wakuv2.test | CHANGED | rc=0 >>
      --eth-client-address=ws://linux-01.he-eu-hel1.nimbus.prater.wg:9547
node-01.ac-cn-hongkong-c.wakuv2.test | CHANGED | rc=0 >>
      --eth-client-address=ws://linux-01.he-eu-hel1.nimbus.prater.wg:9547

staheri14 commented 2 years ago

Okay. Don't you think the flag should indicate in the help message that it expects a websocket?

You are right, although the example address in the help message is a WebSocket address, it is not sufficient and needs to be explicitly stated as well. FYI, created the PR https://github.com/status-im/nwaku/pull/1176 to address this.

staheri14 commented 2 years ago

Fixed:

status-im/infra-nimbus@fa3bb886 - nimbus.prater: open websocket port for wakuv2.test

status-im/infra-nimbus@1f54ea1f - nimbus.prater: open websocket port to other hosts

725ff4a7 - wakuv2-test: use websocket port for eth client
 > a all -a 'grep eth-client /docker/nim-waku-v2/docker-compose.yml' 
node-01.do-ams3.wakuv2.test | CHANGED | rc=0 >>
      --eth-client-address=ws://linux-01.he-eu-hel1.nimbus.prater.wg:9547
node-01.gc-us-central1-a.wakuv2.test | CHANGED | rc=0 >>
      --eth-client-address=ws://linux-01.he-eu-hel1.nimbus.prater.wg:9547
node-01.ac-cn-hongkong-c.wakuv2.test | CHANGED | rc=0 >>
      --eth-client-address=ws://linux-01.he-eu-hel1.nimbus.prater.wg:9547

Thanks a lot for updating the addresses!