Make picoquic_sample work properly under long delay senarios

[KathyNJU]

The purpose of my experiment is to achieve normal file transfer function under long delay senarios (for example, RTT of 2min). As the default handshake_completion_timer was set to 30s and the default maximum_idle_timer was set to 120s, the first trial did not work. According to your guidance, I have made the following modifications. Modification 1: Set the idle timeout parameter for the context (both on server side and client side)

...
// simply call the API (Modification 1)
// set_rtt is the delay introduced by netem, in our experiment is 120s
picoquic_set_default_idle_timeout(quic, (3*set_rtt));

picoquic_set_default_congestion_algorithm(quic, picoquic_bbr_algorithm);
...

Modification 2: set PICOQUIC_MICROSEC_HANDSHAKE_MAX (defined in picoquic_internal.h) to 360000000 (namely 3*set_rtt)

So far, the modifications to maximum_idle_timer and handshake_completion_timer should be completed. However, it still did not work. I also noticed the remarks in Sample: Server or client close the connection if it remains inactive for more than 10 seconds. So I guess it was because the maximum time remained inactive was too short that the handshake failed. As a result, I have made the following modification: Modification 3: set PICOQUIC_MICROSEC_STATELESS_RESET_INTERVAL_DEFAULT (defined in picoquic_internal.h) to 3600000ull.

After making the above modifications, the handshake was not implemented successfully. I was a little lost in the face of the problem. Last time, you mentioned some parameters to overcome the flow control limits. Are these parameters relevant to the handshake?

[KathyNJU]

Great! Thank you for your response! We have now essentially succeeded in deployment and should be able to carry out actual tests soon. If the issue persists, it might be as you suggested; otherwise, it could be some configuration issues in my simulation. I will share the experimental results with you later :)

[KathyNJU]

In picoquic_internal.h, I have noticed that currently there are the following parameters: `

define PICOQUIC_MAX_PACKET_SIZE 1536

define PICOQUIC_MIN_SEGMENT_SIZE 256

define PICOQUIC_ENFORCED_INITIAL_MTU 1200

define PICOQUIC_PRACTICAL_MAX_MTU 1440

define PICOQUIC_RETRY_SECRET_SIZE 64

define PICOQUIC_RETRY_TOKEN_PAD_SIZE 26

` I would like to ask about their specific meanings. The default Ethernet MTU is 1500 bytes, and I'm curious about how the values 1536 and 1440 came about. I looked into RFC 9000, which mentions that in QUIC, the minimum size for link datagrams is supposed to be 1200 bytes (which corresponds to PICOQUIC_ENFORCED_INITIAL_MTU).

I am currently facing a practical deployment issue. We want to conduct actual tests, but our current support for USLP payload length is only 861 bytes, requiring the IP packet size to be less than or equal to 855 bytes. However, this contradicts the stipulations in RFC 9000. Can we still implement QUIC transmission under these conditions? If I want to implement QUIC transmission on the current USLP basis, which parameters do I need to modify?

Additionally, as our current bandwidth can only achieve about 6000bit/s, I think the following two parameters also need to be modified: `

define PICOQUIC_CWIN_INITIAL (10 * PICOQUIC_MAX_PACKET_SIZE)

define PICOQUIC_CWIN_MINIMUM (2 * PICOQUIC_MAX_PACKET_SIZE)

`

Given the indeed small current bandwidth, I think it can be modified to:

`

define PICOQUIC_CWIN_INITIAL (1 * PICOQUIC_MAX_PACKET_SIZE)

define PICOQUIC_CWIN_MINIMUM (1 * PICOQUIC_MAX_PACKET_SIZE)

or

define PICOQUIC_CWIN_MINIMUM (0 * PICOQUIC_MAX_PACKET_SIZE)

`

Or can the existing implementation achieve automatic fragmentation?

[huitema]

yes, I should add comments and document these variables:

#define PICOQUIC_MAX_PACKET_SIZE 1536
This is the memory allocated in the packet buffer. Sending more than that is impossible.

#define PICOQUIC_MIN_SEGMENT_SIZE 256
To avoid trying to send coalesced packets if the remaining space in sending buffer is too small

#define PICOQUIC_ENFORCED_INITIAL_MTU 1200
Per RFC 9000

#define PICOQUIC_PRACTICAL_MAX_MTU 1440
Based on current Internet observation, MTU remaining after commonly used tunnels, etc.

#define PICOQUIC_RETRY_SECRET_SIZE 64
Max length that can configured for the secret protecting the retry packets -- worst case 512 bits

#define PICOQUIC_RETRY_TOKEN_PAD_SIZE 26
Tokens are padded to that length to avoid leaking information through token length

To answer your other question, no you cannot use an RFC 9000 compliant version of QUIC if the network cannot carry packets with the minimal IPv6 MTU. You could of course change the variable and recompile the code, but you would not interoperate with other implementations of QUIC.

[KathyNJU]

So, this means that if I want to implement QUIC communication on the existing USLP basis, I could modify the parameters like this:

#ifndef PICOQUIC_MAX_PACKET_SIZE
#define PICOQUIC_MAX_PACKET_SIZE 861
#endif
#define PICOQUIC_MIN_SEGMENT_SIZE 256
#define PICOQUIC_ENFORCED_INITIAL_MTU 700
#define PICOQUIC_PRACTICAL_MAX_MTU 855
#define PICOQUIC_RETRY_SECRET_SIZE 64
#define PICOQUIC_RETRY_TOKEN_PAD_SIZE 26
#define PICOQUIC_DEFAULT_0RTT_WINDOW (10*PICOQUIC_ENFORCED_INITIAL_MTU)

Are there any other parameters that need to be modified?

Do you mean that after these modifications, Picoquic will still be able to transmit successfully, albeit not in compliance with RFC9000, and it won't interact with other QUIC implementations (like MsQuic), but it can interact with my modified Picoquic implementation? Is my understanding correct?

[huitema]

Yes, that should work. Of course, the best is to program the parameters of your network in Mininet or something similar, and try. 6000bps is very low, lower than most existing tests, so I am not completely sure that everything will work.

You should be careful with the computation of packet length. Picoquic is interested with the size of QUIC packets, i.e., the size of the UDP payload. The complete packet may have 20 bytes of IPv4 header, and 16 bytes of UDP payload. If you send a QUIC packet of 861 bytes, you may end up sending 861 + 16 + 20 = 897 bytes. Is that what you want?

If you do know the packet size, you could also set the parameters to the exact values that you expect, and completely bypass PMTU discovery.

[KathyNJU]

Before modifying the MTU, I tested at 6000bps, and it was successful in file transfer. After modification, I will conduct actual tests.

In the practical tests, I used IPv6, and the packet size of 861 bytes I mentioned specifically refers to: 861 = 5 (encapsulation header) + 40 (IPv6 header) + 8 (UDP header) + user data.

In that case, should my PICOQUIC_MAX_PACKET_SIZE be set to 807?

I'm also concerned that Picoquic's own packet headers will be encrypted. I would like to know how parameters such as PICOQUIC_ENFORCED_INITIAL_MTU and PICOQUIC_PRACTICAL_MAX_MTU are calculated. I'm worried that incorrect settings might prevent normal encryption and other processes, causing Picoquic to malfunction.

[huitema]

You could set PICOQUIC_MAX_PACKET_SIZE, PICOQUIC_ENFORCED_INITIAL_MTU and PICOQUIC_PRACTICAL_MAX_MTU to 807. That would disable PMTU discovery, one less thing to worry about.

[KathyNJU]

#ifndef PICOQUIC_MAX_PACKET_SIZE
#define PICOQUIC_MAX_PACKET_SIZE 807
#endif
#define PICOQUIC_MIN_SEGMENT_SIZE 256
#define PICOQUIC_ENFORCED_INITIAL_MTU 807
#define PICOQUIC_ENFORCED_INITIAL_CID_LENGTH 8
#define PICOQUIC_PRACTICAL_MAX_MTU 807
#define PICOQUIC_RETRY_SECRET_SIZE 64
#define PICOQUIC_RETRY_TOKEN_PAD_SIZE 26
#define PICOQUIC_DEFAULT_0RTT_WINDOW (7*PICOQUIC_ENFORCED_INITIAL_MTU)
#define PICOQUIC_NB_PATH_TARGET 8
#define PICOQUIC_NB_PATH_DEFAULT 2
#define PICOQUIC_MAX_PACKETS_IN_POOL 0x2000
#define PICOQUIC_STORED_IP_MAX 16

Do I need to make corresponding modifications to parameters like PICOQUIC_MIN_SEGMENT_SIZE, PICOQUIC_RETRY_SECRET_SIZE, PICOQUIC_RETRY_TOKEN_PAD_SIZE, and PICOQUIC_MAX_PACKETS_IN_POOL?

Considering the link bandwidth is 6000bps, should PICOQUIC_DEFAULT_0RTT_WINDOW be modified as mentioned above?

[KathyNJU]

However, the initial value of PICOQUIC_MAX_PACKET_SIZE is 1536, while the MTU for Ethernet is 1500. Does this include other things as well?

The initial PICOQUIC_PRACTICAL_MAX_MTU is 1440. Is this because it's calculated as 1500 (Ethernet MTU) - 8 (UDP header) - 40 (IPv6 header)?

[huitema]

This just shows how old I am. Ethernet V1, as deployed in the 1980's, had an MTU of 1536 bytes. V2 is 1500 bytes. It does not matter very much, but the first value tried for PMTU discovery does matter -- too high and the first trial is lost. Look at https://en.wikipedia.org/wiki/Maximum_transmission_unit for series of plausible values. 1500 bytes will not work for residential customers if the path between the home router and the ISP uses PPPoE (1492), or if one the path is some kind of tunnel or VPN. 1440 is just a value that happens to work with a majority of these tunneling techniques.

[KathyNJU]

OK, thank you very much for your explanation!