informalsystems / tendermint-rs

Client libraries for Tendermint/CometBFT in Rust!
Apache License 2.0
615 stars 225 forks source link

Websocket client is unable to subscribe to vote events #1432

Open bonedaddy opened 6 months ago

bonedaddy commented 6 months ago

What went wrong?

I'm unable to subscribe to events of type Vote when using WebsocketClient. I have tested against the rpc node using websocat, which correctly prints out votes. However I'm unable to achieve the same functionality using tendermint_rpc.

$> echo '{ "jsonrpc": "2.0","method": "subscribe","id": 0,"params": {"query": "tm.event='"'Vote'"'"} }' | websocat -n -t ws://<RPC>/websocket

Steps to reproduce

This is the code I have that acts as a wrapper around WebsocketClient https://github.com/rangesecurity/ctop/blob/main/ws_client/src/lib.rs

To test clone the repository locally:

$> git clone https://github.com/rangesecurity/ctop
$> cd ctop
$> cargo build --bin cli
$> ./target/debug/cli subscirbe --url <RPC_NODE>

Nothing gets printed to stdout, when the expected result are consensus votes being printed. For reference if I change this line to "tm.event='NewBlock'" I correctly receive events for new blocks.

Definition of "done"

Able to subscribe to vote events ,as well as other custom events (NewRound, NewRoundStep)

romac commented 6 months ago

Thanks for the report! Sounds like the same issue as https://github.com/informalsystems/tendermint-rs/issues/1426

bonedaddy commented 6 months ago

Let me try that, if it is I will close this issue since it's a duplicate.

bonedaddy commented 6 months ago

@romac so I tried the branch you linked in #1426 and didn't receive any vote events, however when I enable debug logging, it does seem the WebSocketClient is indeed receiving the events, they just aren't being sent to the caller:

2024-05-31T07:30:31.885937Z DEBUG tendermint_rpc::client::transport::websocket: /home/solidity/.cargo/git/checkouts/tendermint-rs-7b2e4433a0119a31/b1aa9fd/rpc/src/client/transport/websocket.rs:922: Generic JSON-RPC message: Wrapper { jsonrpc: Version("2.0"), id: Str("ec4063c7-4627-47ab-8aa5-42ddc8dc622e"), result: Some(GenericJsonResponse(Object {"data": Object {"type": String("tendermint/event/Vote"), "value": Object {"Vote": Object {"block_id": Object {"hash": String("24580073F7C36B063B98C65DCC2AB3312745851D85182BFEDF2DDECC17A07C2B"), "parts": Object {"hash": String("A0A5F3DDC7C51F9CD0B6D5C1A2426A88346F21D5C560EC715DD913308756A42E"), "total": Number(2)}}, "height": String("16250129"), "round": Number(0), "signature": String("kC9K7S0uv1BJPMhvNlqMGxTfPYSI5n4TqzLAWYGRhq+LOlKAxenbdfoGDZpK9dJ5JWmOd6XwVVEMlZm/t7PWDA=="), "timestamp": String("2024-05-31T07:30:31.613584288Z"), "type": Number(1), "validator_address": String("DD5751613FD7D31A952353014BD39FF5609CE2AF"), "validator_index": Number(149)}}}, "events": Object {"tm.event": Array [String("Vote")]}, "query": String("tm.event = 'Vote'")})), error: None }
2024-05-31T07:30:31.886028Z DEBUG tendermint_rpc::client::transport::websocket: /home/solidity/.cargo/git/checkouts/tendermint-rs-7b2e4433a0119a31/b1aa9fd/rpc/src/client/transport/websocket.rs:922: Generic JSON-RPC message: Wrapper { jsonrpc: Version("2.0"), id: Str("ec4063c7-4627-47ab-8aa5-42ddc8dc622e"), result: Some(GenericJsonResponse(Object {"data": Object {"type": String("tendermint/event/Vote"), "value": Object {"Vote": Object {"block_id": Object {"hash": String("24580073F7C36B063B98C65DCC2AB3312745851D85182BFEDF2DDECC17A07C2B"), "parts": Object {"hash": String("A0A5F3DDC7C51F9CD0B6D5C1A2426A88346F21D5C560EC715DD913308756A42E"), "total": Number(2)}}, "height": String("16250129"), "round": Number(0), "signature": String("AckztHXWteFrF0I1chGFiSjGMo1nk88wsKJ5NeBBZKMASTx2frY8P9EYfhYhWxBsbFJrpr0TXseMefnleRruDw=="), "timestamp": String("2024-05-31T07:30:31.65383738Z"), "type": Number(1), "validator_address": String("583AE736E67DF9D72FE87B9AA7D3210D3B4B0E5A"), "validator_index": Number(116)}}}, "events": Object {"tm.event": Array [String("Vote")]}, "query": String("tm.event = 'Vote'")})), error: None }

I pushed a new branch to my codebase so this can be replicated https://github.com/rangesecurity/ctop/tree/debug/romac

romac commented 6 months ago

Ah sorry, it's actually a different problem.

We do no have support for Vote events in tendermint-rs at the moment, only Tx and NewBlock.

I can add support for those, will keep you posted.

romac commented 6 months ago

Can you give this branch a try and see if it works now?

bonedaddy commented 6 months ago

Can you give this branch a try and see if it works now?

Does not seem to work :thinking:

romac commented 6 months ago

Perhaps this also needs https://github.com/informalsystems/tendermint-rs/pull/1433, I'll take a closer look. Might have to wait until next week unfortunately. But feel free to dig deeper in the meantime!

bonedaddy commented 6 months ago

Perhaps this also needs #1433, I'll take a closer look. Might have to wait until next week unfortunately. But feel free to dig deeper in the meantime!

I forked tendermint-rs and merged romac/router-query into romac/missing-event-types and tried again but also didnt work :thinking: Will poke around to see if i can help.

Branch i forked

edit:

I've isolated the issue to self.pending_commands.remove not running. On line 922 of transport/websoocket.rs I added the following logi

        if let Some(pending_cmd) = self.pending_commands.remove(&id) {
            self.respond_to_pending_command(pending_cmd, msg).await?;
        } else {
            panic!("failed to find pending command");
        }

which is causing my program to panic, so it seems that the removal of the pending command is not working