[RPC] Can not start relayer

ZZzyc001 commented 6 months ago

Roller version

v1.0.6-beta

OS

Ubuntu 22.04.3 LTS

Processor Architecture

x86_64

Others

No response

Steps to reproduce it

I run the commands as follows:

sudo systemctl kill sequencer
sudo systemctl kill relayer
roller config set hub-rpc <rpc-url-with-port>
sudo systemctl start sequencer
sudo systemctl start relayer
sudo systemctl status sequencer
sudo systemctl status relayer

, where the is tried among the following URLs:

https://rpc.dymension-froopyland-2.silentvalidator.com:443
https://froopyland.rpc.silknodes.io:443
https://rpc-dymension-testnet.mzonder.com:443
https://jsonrpc.dymension.nodestake.org:443
https://jsonrpc-dymension.mzonder.com:443
https://dymension-evm.blockpi.network:443/v1/rpc/public

.

Expected result

After executing

sudo systemctl status sequencer

, it should be

● sequencer.service - Roller sequencer service Loaded: loaded (/etc/systemd/system/sequencer.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2024-02-19 14:05:02 CST; 14s ago Main PID: 479500 (roller) Tasks: 12 (limit: 18315) Memory: 201.2M CPU: 1.356s CGroup: /system.slice/sequencer.service ├─479500 /usr/local/bin/roller sequencer start └─479519 /usr/local/bin/rollapp_evm start --home /root/.roller/rollapp --log-file /root/.roller/rollapp/rollapp.log

Feb 19 14:05:02 iZt4ne0x8ux62yjulmbfrgZ systemd[1]: Started Roller sequencer service. Feb 19 14:05:03 iZt4ne0x8ux62yjulmbfrgZ roller[479500]: 💈 The Rollapp sequencer is running on your local machine! Feb 19 14:05:03 iZt4ne0x8ux62yjulmbfrgZ roller[479500]: 💈 Endpoints: Feb 19 14:05:03 iZt4ne0x8ux62yjulmbfrgZ roller[479500]: 💈 EVM RPC: http://0.0.0.0:8545 Feb 19 14:05:03 iZt4ne0x8ux62yjulmbfrgZ roller[479500]: 💈 Node RPC: http://0.0.0.0:26657 Feb 19 14:05:03 iZt4ne0x8ux62yjulmbfrgZ roller[479500]: 💈 Rest API: http://0.0.0.0:1317 Feb 19 14:05:03 iZt4ne0x8ux62yjulmbfrgZ roller[479500]: 💈 Log file path: /root/.roller/rollapp/rollapp.log Feb 19 14:05:03 iZt4ne0x8ux62yjulmbfrgZ roller[479500]: 💈 Rollapp root dir: /root/.roller/rollapp , and after executing
sudo systemctl status relayer
, it should be 💈 The relayer is running successfully on you local machine on channel channel-0!

Actual result

With the use of either one in

https://rpc.dymension-froopyland-2.silentvalidator.com:443
https://froopyland.rpc.silknodes.io:443
https://rpc-dymension-testnet.mzonder.com:443

, it returns similarly as

● relayer.service - Roller relayer service Loaded: loaded (/etc/systemd/system/relayer.service; enabled; vendor preset: enabled) Active: activating (auto-restart) (Result: exit-code) since Mon 2024-02-19 14:21:54 CST; 520ms ago Process: 482057 ExecStart=/usr/local/bin/roller relayer start (code=exited, status=1/FAILURE) Main PID: 482057 (code=exited, status=1/FAILURE) CPU: 294ms

after

● relayer.service - Roller relayer service Loaded: loaded (/etc/systemd/system/relayer.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2024-02-19 14:05:23 CST; 12min ago Main PID: 479541 (roller) Tasks: 14 (limit: 18315) Memory: 65.7M CPU: 15.661s CGroup: /system.slice/relayer.service ├─479541 /usr/local/bin/roller relayer start └─479648 /usr/local/bin/roller_bins/rly tx channel -t 60s -d --override rollapp-hub --home /root/.roller/relayer

Feb 19 14:05:23 iZt4ne0x8ux62yjulmbfrgZ systemd[1]: Started Roller relayer service. Feb 19 14:05:25 iZt4ne0x8ux62yjulmbfrgZ roller[479541]: 💈 Establishing IBC transfer channel Feb 19 14:05:52 iZt4ne0x8ux62yjulmbfrgZ roller[479541]: 💈 Creating channel...

With the same setup, it would try to restart. By using the command

sudo journalctl -u relayer -f

, it follows

relayer.service: Main process exited, code=exited, status=1/FAILURE relayer.service: Failed with result 'exit-code'. relayer.service: Scheduled restart job, restart counter is at 288. Stopped Roller relayer service. Started Roller relayer service. 💈 command execution failed: exit status 1, stderr: Error: error in json rpc client, with http response metadata: (Status: 503 Service Unavailable, Protocol HTTP/1.1). error unmarshalling: invalid character 'o' in literal null (expecting 'u')

By using the command

tail -f ~/.roller/relayer/relayer.log

, it follows

error Failed to query node status {"chain_name": "fhgs_7298220-1", "chain_id": "fhgs_7298220-1", "attempt": 2, "max_attempts": 5, "error": "failed to query node status: error in json rpc client, with http response metadata: (Status: 503 Service Unavailable, Protocol HTTP/1.1). error unmarshalling: invalid character 'o' in literal null (expecting 'u')"} debug Queried block {"chain_name": "froopyland_100-1", "chain_id": "froopyland_100-1", "height": 2644867, "latest": 2644867, "delta": 0} debug Retaining packet message {"chain_name": "froopyland_100-1", "chain_id": "froopyland_100-1", "event_type": "send_packet", "sequence": 6843, "channel_id": "channel-2725", "port_id": "transfer", "counterparty_channel_id": "channel-1", "counterparty_port_id": "transfer"} debug Observed IBC message {"chain_name": "froopyland_100-1", "chain_id": "froopyland_100-1", "event_type": "send_packet", "sequence": 6843, "src_channel": "channel-2725", "src_port": "transfer", "dst_channel": "channel-1", "dst_port": "transfer", "timeout_height": 68533, "timeout_height_revision": 1, "timeout_timestamp": 1708324068380376988} debug Retaining packet message {"chain_name": "froopyland_100-1", "chain_id": "froopyland_100-1", "event_type": "send_packet", "sequence": 200882, "channel_id": "channel-8476", "port_id": "transfer", "counterparty_channel_id": "channel-0", "counterparty_port_id": "transfer"} debug Observed IBC message {"chain_name": "froopyland_100-1", "chain_id": "froopyland_100-1", "event_type": "send_packet", "sequence": 200882, "src_channel": "channel-8476", "src_port": "transfer", "dst_channel": "channel-0", "dst_port": "transfer", "timeout_height": 7533, "timeout_height_revision": 1, "timeout_timestamp": 1708324069134291840} error Failed to query node status {"chain_name": "fhgs_7298220-1", "chain_id": "fhgs_7298220-1", "attempt": 3, "max_attempts": 5, "error": "failed to query node status: error in json rpc client, with http response metadata: (Status: 503 Service Unavailable, Protocol HTTP/1.1). error unmarshalling: invalid character 'o' in literal null (expecting 'u')"} error Failed to query node status after max attempts {"chain_name": "fhgs_7298220-1", "chain_id": "fhgs_7298220-1", "attempts": 5, "error": "context deadline exceeded"} error Failed to query node status after max attempts {"chain_name": "fhgs_7298220-1", "chain_id": "fhgs_7298220-1", "attempts": 5, "error": "context deadline exceeded"} debug Context done, quitting PathProcessor {"chain_id_1": "froopyland_100-1", "chain_id_2": "fhgs_7298220-1", "client_id_1": "07-tendermint-16088", "client_id_2": "07-tendermint-26", "error": "context deadline exceeded"}

With the use of

https://jsonrpc.dymension.nodestake.org:443

, it returns

● relayer.service - Roller relayer service Loaded: loaded (/etc/systemd/system/relayer.service; enabled; vendor preset: enabled) Active: activating (auto-restart) (Result: exit-code) since Mon 2024-02-19 15:09:22 CST; 443ms ago Process: 494680 ExecStart=/usr/local/bin/roller relayer start (code=exited, status=1/FAILURE) Main PID: 494680 (code=exited, status=1/FAILURE) CPU: 143ms

after

Feb 19 14:44:31 iZt4ne0x8ux62yjulmbfrgZ systemd[1]: Started Roller relayer service. Feb 19 14:44:33 iZt4ne0x8ux62yjulmbfrgZ roller[490728]: 💈 Establishing IBC transfer channel Feb 19 14:45:00 iZt4ne0x8ux62yjulmbfrgZ roller[490728]: 💈 Creating channel...

With the same setup, it would try to restart. By using the command

sudo journalctl -u relayer -f

, it follows

relayer.service: Main process exited, code=exited, status=1/FAILURE relayer.service: Failed with result 'exit-code'. relayer.service: Scheduled restart job, restart counter is at 541. Stopped Roller relayer service. Started Roller relayer service. 💈 command execution failed: exit status 1, stderr: Error: error in json rpc client, with http response metadata: (Status: 200 OK, Protocol HTTP/1.1). RPC error -32601 - the method abci_query does not exist/is not available

By using the command

tail -f ~/.roller/relayer/relayer.log

, it follows

error Failed to query node status {"chain_name": "froopyland_100-1", "chain_id": "froopyland_100-1", "attempt": 4, "max_attempts": 5, "error": "failed to query node status: error in json rpc client, with http response metadata: (Status: 200 OK, Protocol HTTP/1.1). RPC error -32601 - the method status does not exist/is not available"} error Failed to query node status {"chain_name": "fhgs_7298220-1", "chain_id": "fhgs_7298220-1", "attempt": 5, "max_attempts": 5, "error": "failed to query node status: error in json rpc client, with http response metadata: (Status: 503 Service Unavailable, Protocol HTTP/1.1). error unmarshalling: invalid character 'o' in literal null (expecting 'u')"} error Failed to query node status after max attempts {"chain_name": "fhgs_7298220-1", "chain_id": "fhgs_7298220-1", "attempts": 5, "error": "failed to query node status: error in json rpc client, with http response metadata: (Status: 503 Service Unavailable, Protocol HTTP/1.1). error unmarshalling: invalid character 'o' in literal null (expecting 'u')"} error Failed to query node status {"chain_name": "fhgs_7298220-1", "chain_id": "fhgs_7298220-1", "attempt": 1, "max_attempts": 5, "error": "failed to query node status: error in json rpc client, with http response metadata: (Status: 503 Service Unavailable, Protocol HTTP/1.1). error unmarshalling: invalid character 'o' in literal null (expecting 'u')"} debug No cached IBC header for client trusted height {"chain_id": "fhgs_7298220-1", "client_id": "07-tendermint-26", "height": 2588415} error Failed to query node status {"chain_name": "fhgs_7298220-1", "chain_id": "fhgs_7298220-1", "attempt": 2, "max_attempts": 5, "error": "failed to query node status: error in json rpc client, with http response metadata: (Status: 503 Service Unavailable, Protocol HTTP/1.1). error unmarshalling: invalid character 'o' in literal null (expecting 'u')"} error Failed to query node status {"chain_name": "fhgs_7298220-1", "chain_id": "fhgs_7298220-1", "attempt": 3, "max_attempts": 5, "error": "failed to query node status: error in json rpc client, with http response metadata: (Status: 503 Service Unavailable, Protocol HTTP/1.1). error unmarshalling: invalid character 'o' in literal null (expecting 'u')"} error Failed to query latest height after max attempts {"chain_name": "froopyland_100-1", "chain_id": "froopyland_100-1", "attempts": 5, "error": "context deadline exceeded"} error Failed to query node status after max attempts {"chain_name": "fhgs_7298220-1", "chain_id": "fhgs_7298220-1", "attempts": 5, "error": "context deadline exceeded"} debug Context done, quitting PathProcessor {"chain_id_1": "froopyland_100-1", "chain_id_2": "fhgs_7298220-1", "client_id_1": "07-tendermint-16088", "client_id_2": "07-tendermint-26", "error": "context deadline exceeded"}

With the use of

https://jsonrpc-dymension.mzonder.com:443

, it returns

● sequencer.service - Roller sequencer service Loaded: loaded (/etc/systemd/system/sequencer.service; enabled; vendor preset: enabled) Active: activating (auto-restart) (Result: exit-code) since Mon 2024-02-19 15:18:13 CST; 1s ago Process: 497036 ExecStart=/usr/local/bin/roller sequencer start (code=exited, status=1/FAILURE) Main PID: 497036 (code=exited, status=1/FAILURE) CPU: 108ms

With the same setup, it would try to restart. By using the command

sudo journalctl -fu sequencer.service

, it follows

systemd[1]: sequencer.service: Main process exited, code=exited, status=1/FAILURE systemd[1]: sequencer.service: Failed with result 'exit-code'. systemd[1]: sequencer.service: Scheduled restart job, restart counter is at 166. systemd[1]: Stopped Roller sequencer service. systemd[1]: Started Roller sequencer service. roller[497348]: 💈 command execution failed: exit status 1, stderr: Error: error in json rpc client, with http response metadata: (Status: 200 OK, Protocol HTTP/1.1). RPC error -32601 - the method abci_query does not exist/is not available

With the use of

https://dymension-evm.blockpi.network:433/v1/rpc/public

, by using the command

sudo journalctl -fu sequencer.service

, it follows

systemd[1]: sequencer.service: Main process exited, code=exited, status=1/FAILURE systemd[1]: sequencer.service: Failed with result 'exit-code'. systemd[1]: sequencer.service: Scheduled restart job, restart counter is at 326. systemd[1]: Stopped Roller sequencer service. systemd[1]: Started Roller sequencer service.

, which would not end.

Relevant log output

No response

Notes

No response

alanwilhelm commented 6 months ago

I have not gotten a response for this either.

https://github.com/dymensionxyz/roller/issues/732

omritoptix commented 6 months ago

looking. thanks.

ZZzyc001 commented 6 months ago

looking. thanks.

I tried again today. Every time I run below sequentially:

sudo systemctl kill sequencer; sudo systemctl kill relayer
systemctl disable relayer; systemctl disable sequencer
roller config set hub-rpc "<rpc-url-with-port>"
systemctl enable relayer; systemctl enable sequencer
sudo systemctl start sequencer; sudo journalctl -fu sequencer.service
sudo systemctl start relayer; sudo journalctl -fu relayer.service

.

With <rpc-url-with-port> in

https://rpc.dymension-froopyland-2.silentvalidator.com:443
https://froopyland.rpc.silknodes.io:443
https://rpc-dymension-testnet.mzonder.com:443
https://jsonrpc.dymension.nodestake.org:443
https://froopyland.blockpi.network:443/rpc/v1/public

, the error occurs about the relayer while the sequencer behaves normally. The error is

💈 Establishing IBC transfer channel 💈 Creating channel... 💈 Validating channel established... 💈 command execution failed: exit status 1, stderr: Error: error in json rpc client, with http response metadata: (Status: 503 Service Unavailable, Protocol HTTP/1.1). error unmarshalling: invalid character 'o' in literal null (expecting 'u')

With <rpc-url-with-port> in

https://jsonrpc-dymension.mzonder.com:443
https://dymension-evm.blockpi.network:433/v1/rpc/public

, the error occurs about the sequencer. The error is

💈 command execution failed: exit status 1, stderr: Error: error in json rpc client, with http response metadata: (Status: 200 OK, Protocol HTTP/1.1). RPC error -32601 - the method abci_query does not exist/is not available

Maybe the above information could help you better.

Tastygeek commented 6 months ago

hi bro,have you finally solved it? Is there a stable rpc?

alanwilhelm commented 6 months ago

Still bricked?

dymensionxyz / roller