search

sctplab / usrsctp

A portable SCTP userland stack
BSD 3-Clause "New" or "Revised" License
664 stars 279 forks source link

snd_ppid == 50 priority #213

Open shiretu opened 6 years ago

shiretu commented 6 years ago

Hi,

We have encountered significant delays on WebRTC data channels opening. Depending on how large the send buffer is for the SCTP socket, the sending of those snd_ppid == 50 messages takes any time from very fast to vey slow.

On slow connections, data channels opening while having ongoing transfers takes even 25 seconds, because all data enqueued has to be sent.

Moreover, on slow connections, we also hit this a lot: (stcb->asoc.chunks_on_out_queue >= SCTP_BASE_SYSCTL(sctp_max_chunks_on_queue) Basically, we should have at least reserved (if not always-available) chunks for control messages.

Is there a way to give absolute priority to those messages so the channels can be opened? I-DATA does not help at all unfortunately.

lgrahl commented 6 years ago

I-DATA does not help at all unfortunately.

I don't think that is correct. The only reason it doesn't help so far is that it simply isn't active on the sender side in usrsctp at the moment.

shiretu commented 6 years ago

I think I was not clear enough. The I-DATA support is active, I have pcap files to prove that. However, it does not even get a chance to do its thing, because usrsctp_sendv bails out with errno == EWOULDBLOCK. The reason for that is not the lack of send space, but the lack of chunk blocks. The test that fails is sctp_max_chunks_on_queue.

Ideally, there should be a way to overshoot those limits or undershoot the limits for normal data and reserve/guarantee sending success for certain message types.

shiretu commented 6 years ago

Currently, there is no way that I'm aware of to do that in the library itself, or provide means to do that at app level. A simple query to return number of buffers available and number of bytes free on the send space should be enough to fix this situation on app level.

tuexen commented 6 years ago

@shiretu I agree, the buffer management has to be improved to allow for some space at each stream for the API used by WebRTC.

Please note that you can change the parameter sctp_max_chunks_on_queue by using usrsctp_sysctl_set_ sctp_max_chunks_on_queue. Not sure if that helps.

shiretu commented 6 years ago

Hi Michael,

It helps. Thanks. Providing API for querying the send space fullness (all interesting counters) would go a very long way for all user land apps. It will enable us to basically do sends without a single EWOULDBLOCK, reserve space for important messages, get the buffered amounts and use them to do bitrate adaptation for video environments, etc.

It will really open a lot of doors

tuexen commented 6 years ago

OK. Interesting. Implementing the support for bufferedAmount is on the ToDo list, since it is required by the W3C Spec. However, this only cover unsent data. It does not include the data buffered for retransmission. The idea is that you can make sure you have always something to send when needed, but don't buffer more than required in the SCTP stack. In combination with the a hook for the onbufferedamountlow event, would that allow you what you want to do? Or do you really need to take into account how much data is buffered for retransmission? Please note that you can always increase the size of the send buffer. It is just a number. So if you want to spend the resources, just increase it.

shiretu commented 6 years ago

We have some very bad environments to satisfy in our app for example. We do app level buffering but few tens of kilobytes in the sctp buffers, translates in 1-2 seconds of video that throws off our bitrate computations because those are not accounted for. Not a very big deal, but in the essence, this API of query the buffer levels can help in issues we don’t even know we have

tuexen commented 6 years ago

Can you specify "very bad environments" in terms of delay, packet loss rate or some other parameters. We are working on an improved error recovery method for SCTP (based on TCP RACK) and that would help us to see if it would improve things...

shiretu commented 6 years ago

We have to deal with 3 main sets of bad envs:

  1. Bitrate constraints. For example, when traffic quota was exceeded over mobile and the mobile operator sets the device into something like 100kbps
  2. Packet drops/lost due to network connectivity issues
  3. 600+ latency on cross continents communication

Those are the categories. In terms of numbers, I will gather this information as soon as possible.

Thanks for the involvement!!!