migalabs / armiarma

Armiarma is a Libp2p open-network crawler with a current focus on Ethereum's CL network
https://monitoreth.io
MIT License
36 stars 13 forks source link

WIP: feat: Publish events via SSE #68

Closed samcm closed 4 months ago

samcm commented 7 months ago

Motivation

Publishes Server Sent Events (SSE) allowing external consumers to listen for events created by Armiarma.

Using https://github.com/r3labs/sse

Description

Creates a new events package, with a Forwarder struct that listens for "internal" events, annotates them with extra data, and published a new event over SSE that is sanitized. Attempting to avoid importing "internal" structs in these events so we have a clear seperation of concerns.

Created 2 initial events:

Tasks

Unknown

Proof of Success

$ curl  "http://localhost:9099/events?stream=timed_ethereum_attestation" -H "accept: text/event-stream"

id:
data: {"attestation":{"aggregation_bits":"0x0000000000000000000000000000000000000000000040000000000000000080","data":{"slot":"7074121","index":"19","beacon_block_root":"0x0f45217aed1a9a492f2b5e39e95b91be3f7c50479ec8b8faed3d49f556974c4d","source":{"epoch":"221065","root":"0x4e5a9cbe289e25dd4b4307ac1040fbb422b5f701165c5670d6e025101d3e642a"},"target":{"epoch":"221066","root":"0xd71fd1327a0f4342b4bb16b2f4b7a519bc96b8df0ec82b26f608ac1c93ba7e70"}},"signature":"0x877e8383053985855862de5e581529c5c609626cbdaf9df58cee4dc47efb8abc9b1a86e69584c80d8ea74ca15c14a50404541654078113a85f07e83664786e4f9df14b8a73e2a7a7fd02937e442161fc5f3d1904737fa3bc2845b95851a614ce"},"attestation_extra_data":{"arrived_at":"2023-12-01T02:24:23.201281754Z","peer_msg_id":"OpxZmqvTwl6qGZpfDUvxL1i0Q5AbvFPSXxdhEz4DNbg=","subnet":19,"time_in_slot":9683988201281754},"peer_info":{"id":"\u0000%\u0008\u0002\u0012!\u0003oђ\u0004\ufffdu\u0013r2\ufffd\ufffd\ufffdn\ufffd\ufffd\ufffd4\ufffdn\u00029\ufffdJ\ufffd#\u000f\u0008i\u0016\ufffd\u000cv","ip":"150.136.47.40","port":9000,"user_agent":"Lighthouse/v4.5.0-441fc16/x86_64-linux","latency":276000000,"protocols":["/eth2/beacon_chain/req/ping/1/ssz_snappy","/eth2/beacon_chain/req/status/1/ssz_snappy","/meshsub/1.0.0","/eth2/beacon_chain/req/metadata/1/ssz_snappy","/ipfs/id/push/1.0.0","/eth2/beacon_chain/req/beacon_blocks_by_root/2/ssz_snappy","/eth2/beacon_chain/req/beacon_blocks_by_root/1/ssz_snappy","/eth2/beacon_chain/req/goodbye/1/ssz_snappy","/ipfs/id/1.0.0","/eth2/beacon_chain/req/beacon_blocks_by_range/2/ssz_snappy","/eth2/beacon_chain/req/metadata/2/ssz_snappy","/eth2/beacon_chain/req/beacon_blocks_by_range/1/ssz_snappy","/meshsub/1.1.0"],"protocol_version":"eth2/1.0.0"}}
cortze commented 4 months ago

Hey @samcm, I'm so sorry to come late to check this, but it looks really good! I'll be working on a refactor/update of the crawler over this and the next week, so I'll try to extend the number of stream events as we previously talked