xtaci / smux

A Stream Multiplexing Library for golang with least memory usage(TDMA)
MIT License
1.27k stars 189 forks source link

Blocked streams under heavy load #69

Open netvipe opened 4 years ago

netvipe commented 4 years ago

smux starts blocking/dropping segments when under heavy load. When forwarding rate is below my link rate (>100 Mbps) everything is fine. Once the rate goes above 100 Mbps, smux starts to block exisitng and new streams. This applies to client and to server role.

I've spent too much time with tracing and debugging my code and finally decided to switch back to yamux. Problem disappeared immediatelly.

Please support.

xtaci commented 4 years ago

which protocol version are you using, 1 or 2?

xtaci commented 4 years ago

if we've reached the bandwidth limit,then:

  1. the bandwidth shaper starts to work,streams have priority to send data, may block temporarily for old streams.
  2. the receiver buffer may be full, or the sender-side streams block on writing, if receiver-side streams don't take away their data immediately. (try increase smuxbuf and streambuf(protocol version 2)).
netvipe commented 4 years ago

I'm using version 1.

It's not only that the buffer is full, the whole connection state becomes undefined. The client role return status IsClosed = false even though the peer and underlying transport is fully disconnected.

I'll try with version 2. Let's see how it behaves under my traffic conditions.

xtaci commented 4 years ago

https://github.com/xtaci/smux/issues/59

netvipe commented 4 years ago

Tested the following:

Adaptive peer window makes the difference. I'll test a bit more with version 2.

Many thanks!