search

interledger / interledger-rs

An easy-to-use, high-performance Interledger implementation written in Rust
http://interledger.rs
Other
201 stars 70 forks source link

Improve the high-load of the settlement engine #203

Closed dora-gt closed 5 years ago

dora-gt commented 5 years ago

Though I haven't dug into so much about the problem, it seems that the load of the settlement engine is so high. It may be because of outputting a bunch of logs. Need an inspection and to make the reason clear.

emschwartz commented 5 years ago

I am also seeing this issue while running the ETH + XRP example:

image

gakonst commented 5 years ago

Probably has to do with https://github.com/emschwartz/interledger-rs/blob/master/crates/interledger-settlement-engines/src/engines/ethereum_ledger/eth_engine.rs#L240-L254. Will investigate.

emschwartz commented 5 years ago

The problem is that it is trying to scan every single block since the beginning of the blockchain and firing off one HTTP request for each in parallel. 

[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Current block 12829885
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Will fetch txs from block 0 until 12829885
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Scanning history for incoming ETH transactions
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 0
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 1
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 2
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 3
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 4
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 5
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 6
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 7
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 8
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 9
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 10
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 11
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 12
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 13
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 14
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 15
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 16
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 17
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 18
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 19
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 20
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 21
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 22
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 23
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 24
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 25
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 26
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 27
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 28
[2019-08-14T18:06:43Z TRACE interledger_settlement_engines::engines::ethereum_ledger::eth_engine] Getting txs for block 29

We didn't see this before because we only tested it against ganache-cli, which has no history. :man_facepalming:

emschwartz commented 5 years ago

Oops... Sorry Infura!

image

gakonst commented 5 years ago

Can we close this?

emschwartz commented 5 years ago

Fixed in https://github.com/emschwartz/interledger-rs/pull/213