daimo-eth / daimo

Your own bank, on ethereum
https://daimo.com
GNU General Public License v3.0
326 stars 26 forks source link

Measure transaction latencies #1200

Closed nounderline closed 1 day ago

nounderline commented 1 week ago

Measure latency from broadcasting a transaction to its inclusion in a block:

(broadcast tx) -> (first appeareance of a block) -> (block timestamp)

Transaction is sent directly with eth_sendRawTransaction RPC call. All related calls necessary to make a transaction (gas, block number, nonce) were done outside of timing window for higher measurement accuracy.

Total transaction latency:

  • t0 = moment transaction is submitted to RPC = immediately before we call eth_sendTransaction ... t0_a = immediately after eth_sendTransaction success, shows us how long eth_sendTransaction call takes with various providers
  • t_1 = block timestamp
  • t_2 = moment block is visible in RPC
nounderline commented 1 week ago

Test results on Base mainnet via Alchemy in us-east with 12 transactions. See code and methodology here: https://github.com/nounderline/daimo-tx-latency

Broadcasting transaction via RPC takes ~160ms:

┌──────────────┬──────────────┬──────────────┬──────────────┐
│     min      │     min      │     avg      │    stddev    │
│ decimal(8,1) │ decimal(8,1) │ decimal(8,1) │ decimal(8,1) │
├──────────────┼──────────────┼──────────────┼──────────────┤
│        141.0 │        191.7 │        157.7 │         13.5 │
└──────────────┴──────────────┴──────────────┴──────────────┘

Block timestamps are truncated. When block is seen at the beginning of a second, mined timestamp may be set to previous second (see 16331595):

┌──────────────┬─────────────────────────┬──────────────────────────┬────────────────────┐
│ block_number │        rpc_time         │        block_time        │ broadcast_duration │
│    int64     │        timestamp        │         varchar          │       double       │
├──────────────┼─────────────────────────┼──────────────────────────┼────────────────────┤
│     16331581 │ 2024-06-27 01:41:50.296 │ 2024-06-27T01:41:49.000Z │ 191.67602499999975 │
│     16331583 │ 2024-06-27 01:41:54.209 │ 2024-06-27T01:41:53.000Z │  156.0658229999999 │
│     16331585 │ 2024-06-27 01:41:57.568 │ 2024-06-27T01:41:57.000Z │ 158.25386999999864 │
│     16331587 │ 2024-06-27 01:42:02.32  │ 2024-06-27T01:42:01.000Z │ 149.86705200000142 │
│     16331589 │ 2024-06-27 01:42:06.121 │ 2024-06-27T01:42:05.000Z │ 155.39792999999918 │
│     16331591 │ 2024-06-27 01:42:10.153 │ 2024-06-27T01:42:09.000Z │  166.8116289999998 │
│     16331593 │ 2024-06-27 01:42:14.146 │ 2024-06-27T01:42:13.000Z │  141.0376240000005 │
│     16331595 │ 2024-06-27 01:42:18.134 │ 2024-06-27T01:42:17.000Z │ 149.37409599999955 │
│     16331597 │ 2024-06-27 01:42:22.181 │ 2024-06-27T01:42:21.000Z │ 141.34348699999828 │
│     16331599 │ 2024-06-27 01:42:26.307 │ 2024-06-27T01:42:25.000Z │ 160.62780400000338 │
│     16331602 │ 2024-06-27 01:42:32.342 │ 2024-06-27T01:42:31.000Z │ 166.56657899999846 │
│     16331605 │ 2024-06-27 01:42:38.324 │ 2024-06-27T01:42:37.000Z │ 155.00156600000628 │
├──────────────┴─────────────────────────┴──────────────────────────┴────────────────────┤
│ 12 rows                                                                      4 columns │
dcposch commented 3 days ago

Looks solid.

Can we summarize/simplify?

I'd like to get something that looks like this:

Provider submit>block_ts block_ts>rpc total (submit>rpc)
Alchemy 1100ms 1300ms 2400 ± 950ms
Quicknode 1100ms 1400ms 2500 ± 950ms
Chainstack 1100ms 1500ms 2600 ± 950ms
nounderline commented 2 days ago

"Broadcasting transaction via RPC takes ~160ms" > what does this mean? Blocks are every 2s, so if a transaction is submitted the average latency (submit -> block timestamp) should be >1000ms.

It is the time it takes for eth_sendRawTransaction to be executed.

nounderline commented 2 days ago

Here are results for each provider, 12 transactions each:

┌──────────────────────────┬───────────────┬──────────────┬────────────────────┐
│         Provider         │ send>block_ts │ block_ts>rpc │ total (submit>rpc) │
├──────────────────────────┼───────────────┼──────────────┼────────────────────┤
│ Alchemy                  │ 1530 ms       │ 1188 ms      │ 2719 ± 419 ms      │
│ Quicknode                │ 1571 ms       │ 1451 ms      │ 3021 ± 537 ms      │
│ Chainstack               │ 1101 ms       │ 1878 ms      │ 2979 ± 1397 ms     │
└──────────────────────────┴───────────────┴──────────────┴────────────────────┘

Code and data available here: https://github.com/nounderline/daimo-tx-latency/commit/0335c7a56b05bda9a0e545463cf1e28129e2c2df