mdn / content

The content behind MDN Web Docs
https://developer.mozilla.org
Other
9.24k stars 22.5k forks source link

It's not known where the source of 14kb rule came from #28014

Closed imblowfish closed 1 year ago

imblowfish commented 1 year ago

MDN URL

https://developer.mozilla.org/en-US/docs/Web/Performance/How_browsers_work#tcp_slow_start_14kb_rule

What specific section or headline is this issue about?

Tcp slow start and 14kb rule

What information was incorrect, unhelpful, or incomplete?

First I would like to say thanks to the whole mdn team for their contribution and effort

The documentation says a certain rule, according to which the response to tcp by slow start will be 14kb

However, if you look at the rfc protocol, then there is nothing about 14kb, and the specific size of the response. This protocol is used to determine bandwidth and does not guarantee a specific response size. Even more, the following is written in rfc https://www.rfc-editor.org/rfc/rfc2001

Congestion avoidance and slow start require that two variables be
   maintained for each connection: a congestion window, cwnd, and a slow
   start threshold size, ssthresh.  The combined algorithm operates as
   follows:

   1.  Initialization for a given connection sets cwnd to one segment
       and ssthresh to 65535 bytes.

   2.  The TCP output routine never sends more than the minimum of cwnd
       and the receiver's advertised window.

   3.  When congestion occurs (indicated by a timeout or the reception
       of duplicate ACKs), one-half of the current window size (the
       minimum of cwnd and the receiver's advertised window, but at
       least two segments) is saved in ssthresh.  Additionally, if the
       congestion is indicated by a timeout, cwnd is set to one segment
       (i.e., slow start).

   4.  When new data is acknowledged by the other end, increase cwnd,
       but the way it increases depends on whether TCP is performing
       slow start or congestion avoidance.

      If cwnd is less than or equal to ssthresh, TCP is in slow start;
      otherwise TCP is performing congestion avoidance.  Slow start
      continues until TCP is halfway to where it was when congestion
      occurred (since it recorded half of the window size that caused
      the problem in step 2), and then congestion avoidance takes over.

      Slow start has cwnd begin at one segment, and be incremented by
      one segment every time an ACK is received.  As mentioned earlier,
      this opens the window exponentially:  send one segment, then two,
      then four, and so on.  Congestion avoidance dictates that cwnd be
      incremented by segsize*segsize/cwnd each time an ACK is received,
      where segsize is the segment size and cwnd is maintained in bytes.
      This is a linear growth of cwnd, compared to slow start's
      exponential growth.  The increase in cwnd should be at most one
      segment each round-trip time (regardless how many ACKs are
      received in that RTT), whereas slow start increments cwnd by the
      number of ACKs received in a round-trip time.

The article also describes a specific number, despite the fact that 14kb is a lot and you are unlikely to receive a response of at least more kilobytes to a request to some route with an "Ok" response via http, even if you take into account its wrapping in the headers of the entire tcp ip stack

I would also like to note that the implementation of a specific congestion control algorithm depends on the specific operating system and this also contradicts the described 14kb rule, because some version of the OS may use the congestion Windows agorithm

What did you expect to see?

Perhaps you should just remove the description of this rule and the pica accompanying this section. The current description refers to a too low-level algorithm in the tcp protocol, and does so with errors. This can be a problem for novice developers who consider mdn to be the source of truth and do not yet have sufficient knowledge.

Do you have any supporting links, references, or citations?

https://www.rfc-editor.org/rfc/rfc5681 https://www.rfc-editor.org/rfc/rfc2001

Do you have anything more you want to share?

No response

Josh-Cena commented 1 year ago

There are many blog posts on this topic. To pick one: https://endtimes.dev/why-your-website-should-be-under-14kb-in-size/

The maximum size of a TCP packet is 1500 bytes.

This maximum is not set by the TCP specification, it comes from the ethernet standard

imblowfish commented 1 year ago

There are many blog posts on this topic. To pick one: https://endtimes.dev/why-your-website-should-be-under-14kb-in-size/

The maximum size of a TCP packet is 1500 bytes. This maximum is not set by the TCP specification, it comes from the ethernet standard

Yes, I read this article and saw that you refers to it, but let's check what described in this blog and compare it with what described in RFCs

First:

Most web servers TCP slow start algorithm starts by sending 10 TCP packets.

I didn't found where author found rule about 10 TCP packets. I just found that cwnd may be 1, 2, 4 or 10 MSS (maximum segment size)

Let's check in RFC https://www.rfc-editor.org/rfc/rfc5681#section-3.1 and here what cwnd is:

CONGESTION WINDOW (cwnd): A TCP state variable that limits the amount
      of data a TCP can send.  At any given time, a TCP MUST NOT send
      data with a sequence number higher than the sum of the highest
      acknowledged sequence number and the minimum of cwnd and rwnd.

smss:

SENDER MAXIMUM SEGMENT SIZE (SMSS): The SMSS is the size of the
      largest segment that the sender can transmit.  This value can be
      based on the maximum transmission unit of the network, the path
      MTU discovery [RFC1191, RFC4821] algorithm, RMSS (see next item),
      or other factors.  The size does not include the TCP/IP headers
      and options.

and rmss:

RECEIVER MAXIMUM SEGMENT SIZE (RMSS): The RMSS is the size of the
      largest segment the receiver is willing to accept.  This is the
      value specified in the MSS option sent by the receiver during
      connection startup.  Or, if the MSS option is not used, it is 536
      bytes [RFC1122].  The size does not include the TCP/IP headers and
      options.

and algorithm how to determine initial value of cwnd https://www.rfc-editor.org/rfc/rfc5681#section-3.1:

IW, the initial value of cwnd, MUST be set using the following
   guidelines as an upper bound.

   If SMSS > 2190 bytes:
       IW = 2 * SMSS bytes and MUST NOT be more than 2 segments
   If (SMSS > 1095 bytes) and (SMSS <= 2190 bytes):
       IW = 3 * SMSS bytes and MUST NOT be more than 3 segments
   if SMSS <= 1095 bytes:
       IW = 4 * SMSS bytes and MUST NOT be more than 4 segments

We can see that SMSS can be based on the maximum transmission unit and lets say that based on Ethernet as described in https://www.rfc-editor.org/rfc/rfc4821

As an optimization, it may be appropriate to probe at certain common
or expected MTU sizes, for example, 1500 bytes for standard Ethernet,
or 1500 bytes minus header sizes for tunnel protocols.

So, based on SMSS equals to 1500 we see that SMSS > 1095 bytes and <= 2190 then IW will be 3 * 1500 = 4500 bytes. This is our initial congression window size and there can't be more than 3 segments

I can also show that there are no start 10 packets from the tcp server with the following picture (yellow packets), i connected with curl to google.com and we can see that there are no 10 tcp packets here, tls handshake comes, next ACK HTTP return response to GET and FIN

Screenshot from 2023-07-18 14-37-31

You will see the same thing if you enter the request in the address bar, I used curl to disable all browsers and not spoil the output of wireshark with unnecessary packets

And already in this picture it would be possible to see, summing up all the packets for a given connection, that they barely gain 1600 bytes in total

Next thesis:

This maximum is not set by the TCP specification, it comes from the ethernet standard

Here I agree and 1500 MTU really described in RFC https://www.rfc-editor.org/rfc/rfc4821

As an optimization, it may be appropriate to probe at certain common
or expected MTU sizes, for example, 1500 bytes for standard Ethernet,
or 1500 bytes minus header sizes for tunnel protocols.

Next:

Each TCP packet uses 40 bytes in its header — 16 bytes for IP and an additional 24 bytes for TCP

Yes, but why is it not taken into account here that we are unlikely to transmit an empty TCP packet via TCP on our website, but we will send data in it via the HTTP protocol, and taking into account only the size of the headers is not correct here

Here we move on to the next:

That leaves 1460 bytes per TCP packet. 10 x 1460 = 14600 bytes or roughly 14kB!

As I described above, it is not clear where the author took the mandatory sending of 10 packets, I showed in the image above that there are not 10 of them and that in the protocol 10 does not concern the number of packets, but MSS, how much data we can send over the protocol without the need to separate it

And last:

So if you can fit your website — or the critical parts of it — into 14kB, you can save visitors a lot of time — the time it takes for one round trip between them and your website's server.

14Kb is indeed the normal size of the site, only according to the author's calculations above, it is not clear what the problem will be, for example, with a 15Kb site? in 20Kb? At the basis of all his calculations, it is only unclear where the 10 packets came from, which, according to him, are always sent by slow start anyway (I described above that this is not so) equal to the number of min tcp without data + min ip and why he summed it up like that is unclear

Judging by his words, you will always, in any case, with any connection, receive 10 packets in response from the server and a total of 14Kb, but why?

That's incredibly high for some basic connection.

imblowfish commented 1 year ago

Also the main question is why in your image in the documentation and according to the author there will be 10 packets in response, if the protocol says that cwnd changes on the go after each ACK until it reaches the maximum cwnd value, and does not throw 10 packets at once

Well, and yet, the author refers to Wikipedia, where nothing is said about what he writes, but simply his, possibly incorrect understanding of these rules. Could you provide an argument based on more reliable sources like the RFC or some other standards?

imblowfish commented 1 year ago

If, simply, judging by the description of the blog above, it seems that for every ack, 14Kb, 28Kb, etc. are sent to us by slow start for some reason, although judging by the protocol description, everything is quite the opposite, cwnd is set and after each ACK it increases (usually 2 times) until a loss occurs, in which case cwnd will be reduced However, nowhere is it said that in response to an ACK, the server sends us the same data equal to the size of cwnd, this is just the limit of the transmitted data before splitting

This is confirmed by the image above from wireshark And also the basic logic that if we receive a response from the server via a get-request from "OK", then what's the point of sending us something other than this OK, why do we need protocols that reduce the length of the transmitted message, like protobuf, it was possible not to worry about it and do nothing, we will still get at least 14Kb in the answer

There is also a question, what about embedded devices that also transmit data via tcp, but they, for example, have a memory limit of 10Kb or even less, for which there are also separate simplified protocols that also work via TCP

In general, this rule looks illogical

imblowfish commented 1 year ago

And here I found the article https://tylercipriani.com/blog/2016/09/25/the-14kb-in-the-tcp-initial-window/ where the author indicates that you need this cwnd limit only to specify the limit number after which the client should send ack. If we close the connection before this limit by sending less or sending more, then nothing will happen and only the client will send an additional ack to the server as confirmation that it has received data in the amount of the given cwnd and server can send more

yin1999 commented 1 year ago

As most of servers are running on linux and most of the clients are running on Windows/Android(based on linux kernel), so I just query the default tcp initial CWND of Windows and Linux:

Windows (11, should be the same on windows server): 10 MSS

image

Linux (check https://github.com/torvalds/linux/blob/bfa3037d828050896ae52f6467b6ca2489ae6fb1/include/net/tcp.h#L229-L230): 10 MSS

image

And the source code in Linux header just mention RFC 6928, it just increase the min TCP initial CWND to 10 MSS.