search

sctplab / usrsctp

A portable SCTP userland stack
BSD 3-Clause "New" or "Revised" License
673 stars 280 forks source link

setsockopt IPPROTO_SCTP/SCTP_DELAYED_SACK not functional #317

Open shiretu opened 5 years ago

shiretu commented 5 years ago

I'm trying to debug NAT traversal speed issues (see this for more details).

One of the things I have discovered is the insane amount of SACKs sent. Literally every single packet is ACK-ed. It generates enormous amounts of traffic. At some point, the SACKs sending takes a very very very long break. Would very much like to lower down the frequency, of SACKs to something like 1 SACK at 200 packets or twice a second. The R/W buffers are hefty, they can hold a lot of data. The number of buffers is also 128k (usrsctp_sysctl_set_sctp_max_chunks_on_queue)

For this, I have used:

usrsctp_sysctl_set_sctp_sack_freq_default(200);
usrsctp_sysctl_set_sctp_delayed_sack_time_default(500);

Just to be sure, I wanted to print them:

    struct sctp_sack_info delayedAcks;
    delayedAcks.sack_assoc_id = SCTP_ALL_ASSOC;
    len = sizeof (delayedAcks);
    if (usrsctp_getsockopt(_pSCTPSocket, IPPROTO_SCTP, SCTP_DELAYED_SACK,
            &delayedAcks, &len)) 
        return false;
    fprintf(stderr, "sctp_sack_info: Delay: %" PRIu32 "; Freq: %" PRIu32 "\n",
            delayedAcks.sack_delay, delayedAcks.sack_freq);

And the output:

sctp_sack_info: Delay: 500; Freq: 200

Finally, I have written a very small function which looks out for SACKs in the output queue for the receiver of the real data and here is what I see:

T: SELECTIVE_ACK(03); CTSNACK: 0xcd9ce9fe; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9ce9ff; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea00; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea01; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea02; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea03; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea04; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea05; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea06; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea07; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea08; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea09; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea0a; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea0b; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea0c; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea0d; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea0e; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea0f; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea10; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea11; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea12; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea13; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea14; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea15; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0
T: SELECTIVE_ACK(03); CTSNACK: 0xcd9cea16; diff: 1; ARWND: 31997696; GAPACKBLKS: 0; DUPTSNS: 0

Notice how CTSNACK is going up by 1. For clarity, diff is the diff between current CTSNACK and the last sent CTSNACK. I've seen that 4 at the beginning, but never anything else than 1 after that.

tuexen commented 5 years ago

Just as a note: Using such a setting (assuming the receiver will do what it is supposed to do), would result in a very bad performance, since the congestion control on the sender side assumes that it receives a SACK for every other packet.

So the intend for the socket option was to allow to

Of course your suggested configuration should work, although it won't behave like you expect, I guess.

shiretu commented 5 years ago

@tuexen - You were right about the performance. I have basically implemented that delayed selective ack in my app, just to see how it affects performance. It took a deep plunge.

I guess the bug report still stands, but as you said, it will not have the expected outcome when/if fixed. Are there any ways to lower down the SACks count without affecting performance? I suspect that the kernel/bitrate/networkInterfaces/cpu is not very happy when the app sends 20k/sec packets :)