Investigate Event Subscription integration in Tracelistener

EmerisHQ / tracelistener

UNIX named pipes-based real-time state listener for Cosmos SDK blockchains

GNU Affero General Public License v3.0

33 stars 8 forks source link

Investigate Event Subscription integration in Tracelistener #134

Open akc2267 opened 2 years ago

akc2267 commented 2 years ago

Tracelistener currently has to be colocated with a node to read traces from its KV store. As we move towards a more scalable and reliable architecture with multiple third-party failover nodes, we need to rethinking this strict coupling of tracelistener and a chain node.

DoD:

[ ] investigate replacing the KV store traces with Event subscription https://docs.tendermint.com/master/tendermint-core/subscription.html
[ ] investigate if any prominent chains have removed event emissions that we would need to update account and staking balances
[ ] if this integration is feasible, create a high-level list of steps we must do to move to this new architecture
[ ] write up your findings and share with the team on slack

Pitasi commented 2 years ago

In the Tendermint docs it says we have two different ways of getting events. The websocket one is being deprecated in Tendermint 0.36, but cosmos-sdk still is on Tendermint 0.35 so we don't really have a choice, we must use the websockets.

I did a simple test like this with crescent in staging:

# Forward port from crescent to my computer
kubectl port-forward svc/crescent 26657:26657

# Connect to it using a tool called "websocat"
websocat ws://127.0.0.1:26657/websocket

# Send a subscription query 
{ "jsonrpc": "2.0", "method": "subscribe", "id": 1, "params": { "query": "tm.event='NewBlock'" } }

And I got the new block event as requested, attached a file (pretty printed) here: https://app.zenhub.com/files/353632561/dd174941-9eb6-413b-a78e-df7fb7fe2c96/download

akc2267 commented 2 years ago

maybe I'm misunderstanding, but I think this means we must use pure RPC going forward, and not rely on websockets. https://docs.tendermint.com/master/tendermint-core/subscription.html#event-log-api

Pitasi commented 2 years ago

yes, but no one is using this tendermint core version yet

Pitasi commented 2 years ago

Anyway, from a quick look at that json it seems to me that this kind of data is "low level". What I mean is that this json file is (probably) the input for the SDK modules, and the modules can contain other logics to update the underlying application database of the chain 🤔

We can't reimplement the sdk modules logic ourselves, but maybe these events can be used as heuristic to update our "smart RPC cache" though.