search

nervosnetwork / tentacle

A multiplexed p2p network framework that supports custom protocols
https://docs.rs/tentacle
MIT License
54 stars 24 forks source link

revert: revert yamux buffer #281

Closed driftluo closed 3 years ago

driftluo commented 3 years ago

The throughput problem reappeared because of this commit https://github.com/nervosnetwork/tentacle/commit/1ea53e989c8a33313ccda69f093f23b35948d3d0

test case: https://github.com/nervosnetwork/tentacle/pull/213

Although in actual production there will be no sub-substreams above 100 levels, this is indeed a problem. After regression testing, the current code will show a significant throughput drop when it is around 800.

perf ./target/release/examples/throughput_test 800

master branch: CPU consumption of distribute_to_substream function accounts for more than 50% patch branch: CPU consumption problem of distribute_to_substream function cannot be clearly observed, about 2.5%

This is a classic continuous memory and segmented memory overhead problem. In a test scenario under full pressure, such memory issues will be triggered frequently and cause huge overhead.

yangby-cryptape commented 3 years ago

The throughput problem reappeared because of this commit 1ea53e9

I want to request changes for 1ea53e9: .fold(0, |acc, item| acc + item.len()) => .map(HashMap::len).sum()