what is the maximum bitrate lsquic can reach?

recnac-itna commented 4 years ago

Hi, I did a preliminary performance test recently. The way I test is writing data by lsquic_stream_write in on_write callback.

I have test packet_size=200K interval=10ms, and packet_size=50K interval=5ms. The max birate is about 30Mbps.

This result does not satisfy our application scenario. Is it because I use it in the wrong way? And what about your test data? Thanks for your help.

dtikhonov commented 4 years ago

Those packet sizes are very large; 200K is obviously wrong. Did you mean something else?

Is the sender using 100% CPU?

Our tests show that CPU will eventually become a bottleneck. Depending on the kernel version (I am assuming you're running this on Linux), sending UDP packets can take as much as 45% CPU. In other words, maximum throughput depends on how fast your processor is.

recnac-itna commented 4 years ago

Sorry, I may not make it clear. Actually I mean the logic data size from upper application, not the real packet in quic protocol. More specifically, I will pass 200K data buffer to lsquic_stream_write every 10 ms. ssize_t lsquic_stream_write(lsquic_stream_t *stream, const void *buf, size_t len)

I wonder whether this bottleneck is caused by the way I used. I test based on the echo_server demo which used libevent to maintain the event loop. In my understanding, the callback on_write is triggered by the libevent timer (prog_timer_handler->lsquic_engine_process_conns->process_connections->full_conn_ci_tick->process_streams_write_events->lsquic_stream_dispatch_write_events->stream_dispatch_write_events_loop->on_write). So it doesn't reach the 100% performance.

dtikhonov commented 4 years ago

Your understanding of the callback mechanism is correct.

You should be able to peg the CPU on the loopback interface. You're not sending fast enough for some reason.

For comparison, try transferring data using the HTTP client and server and see how fast it goes.

recnac-itna commented 4 years ago

I exclude all my external logic and make a new pure test to observe how fast lsquic can write and read. Specifically, this is how I test:

Keep server lsquic_stream_wantwrite as 1
In the on_write callback, write as many as it can, such as lsquic_stream_write(stream, st_h->buf, 0x1000000)
Client read data in similiar way.

Theoretically, it can reach 1000MB/s, but it can only reach 4MB/s. (4MB/s is also the bottleneck in my real test scenario.) As a comparison, TCP can reach 100MB/s. In theory, QUIC should not be this slow. I have tried to shorten the es_clock_granularity and some other opimizations. But the performance didn't improve.

Is there any problem how I use it?

dtikhonov commented 4 years ago

It is difficult to say whether you are doing something wrong from just the bug report. Nevertheless, 4MB/s does seem quite slow.

How fast can data be transferred using http_server and http_client on the same link?

recnac-itna commented 4 years ago

Actually following your advice, I have read the code of http_server and http_client roughly.

In my understanding, the part of stream reading & writing and event loop is similar to echo_server and echo_client. I didn't find the obvious difference between the two, so I thought the test result is similar. And another problem is that there are so many modes in http_server. I'm a little confused about how to modify the code for the performance test.

Can you give me some instructions about how to modify based on http_server? Or provide a performance program? The performance is important to us. It determines whether we can use lsquic in our product. Thanks for your help.

dtikhonov commented 4 years ago

The echo server and client server as basic examples of the API use. We don't use them often, as we mostly work with HTTP/3 protocol and HTTP server and client.

The HTTP server has two modes: filesystem mode (when -r command-line flag is specified) and interop mode (when -r flag is omitted). For performance testing, I recommend using the interop mode. In this mode, responses are generated instead of being read from the filesystem.

The HTTP client has -K option to discard server response (instead of printing them to stdout).

Basically, a performance test could work like this:

Run HTTP server: ./http_server -c example.com,cert,key -s 127.0.0.1:12345
Run HTTP client: time ./http_client -H example.com -s 127.0.0.1:12345 -K -p /file-1G

This will time how long it takes to transfer a one-gigabyte file over the loopback interface.

The HTTP client has several options that allow to vary load: number of connections, number of requests per connection, and others. Have a look at the output of http_client -h.

recnac-itna commented 4 years ago

Thanks. I will make a performance test on my Android device and give you feedback tomorrow.

recnac-itna commented 4 years ago

As you suggested, I made a performance test by http_server and http_client. Test environment is that two Android devices transfer 1G files through WiFi hotspot. The average transmission speed is 8.2MB/s. By contrast, TCP can reach 15.4MB/s. The CPU usage is 6.8% which doesn't seem to be a bottleneck. This is a result from time: 2m06.10s real 0m24.26s user 0m47.01s system One thing I notice is that the time of user+system is about half of real and the lsquic speed is also half of tcp. Is it possible that lsquic is in idle state half of the time and does not make full use of it？

dtikhonov commented 4 years ago

Is it possible that lsquic is in idle state half of the time and does not make full use of it?

Yes, this is exactly what it looks like. There are a few possible explanations:

There is a problem with lsquic's congestion controller. This is unlikely: our testing shows that, given enough CPU, QUIC should always match TCP speed -- in other words, utilize all available bandwidth. On the other hand, it's not impossible: what are the other network characteristics -- for example, RTT?
UDP traffic may be treated differently than TCP traffic by the hotspot.
Some quirk in the Android environment. Did you have to use threads for this test or did you use vanilla http_client and http_server?

recnac-itna commented 4 years ago

The code version and options in http_server and http_client are exactly same as the origin. And android runtime is very similar to Linux. The only difference is I use ndk to build arm64-v8a platform.
I also made a common UDP performance test. The average speed is 14MB/s, which is very close to tcp. So maybe it is not the UDP traffic problem.
The way I test on Android is that execute the executable file directly by adb shell. So the performance should be the best.

recnac-itna commented 4 years ago

For the more network characteristics, in our product, the most important factor we care about is the time-lag. Because we use QUIC to transport video stream.

In our test, we found the lag is too high and not stable. The delay will float in tens to hundreds of milliseconds. And an unacceptable disadvantage is that, from the perspective of the phenomenon, sometimes the data packages will be stuck for a while (a few seconds), and then resume and catching up, like a pulse.

This long and nonuniform delay has a great impact on our video stream application.

dtikhonov commented 4 years ago

You did not answer a couple of my questions:

What is the RTT in your setup?
Are you using modified lsquic code?

dtikhonov commented 3 years ago

In the last few months, we fixed several performance-related bugs in lsquic. I would be interested to learn whether the updated version of the library performs better in your testing.

ghost commented 3 years ago

@recnac-itna 你好，我现在使用的 lsquic 2.30.0版本。遇到的一系列问题和您去年遇到的这些问题基本一致。我想请教一下，你们现在解决这些问题了吗？希望能够在空闲的时候回复一下，谢谢

Wizmann commented 3 years ago

similar issue with perf_server / perf_client on WIndows. I can reach roughly 10MB/s+ throughtput on localhost which is a little bit low.

TYoung86 commented 2 years ago

I am also encountering this. It may be my implementation.

Can you recommend a performance test scenario for windows?

Perhaps some engine settings that can provide best throughput?

TYoungSL commented 2 years ago

@dtikhonov e.g. what parameters to use with perf_server / perf_client to peg cpu on localhost?

TYoungSL commented 2 years ago

Windows:

> perf_client -p 104857600:104857600 -s ::1:443 -H localhost -L notice
10:17:40.678 [NOTICE] conn-stats: connection DFE672238E9D8E52 marked busy: it's had 16 immediate ticks in a row
10:17:40.678 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 13; cwnd: 77679; conn flow: max: 1572864, avail: 1399290; packets: sent: 138, lost: 4, retx: 4, rcvd: 16; batch: count: 11; min: 1; max: 54; avg: 20.88
10:17:41.696 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 983; cwnd: 86020; conn flow: max: 11869489, avail: 1547311; packets: sent: 7137, lost: 2, retx: 2, rcvd: 1031; batch: count: 591; min: 1; max: 58; avg: 6.12
10:17:42.761 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 494; cwnd: 86020; conn flow: max: 15032877, avail: 908348; packets: sent: 2681, lost: 2, retx: 2, rcvd: 499; batch: count: 291; min: 1; max: 15; avg: 5.50
10:17:43.823 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 534; cwnd: 86020; conn flow: max: 19777959, avail: 1511811; packets: sent: 2919, lost: 2, retx: 2, rcvd: 544; batch: count: 320; min: 1; max: 15; avg: 6.42
10:17:44.891 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 583; cwnd: 86020; conn flow: max: 23735010, avail: 938167; packets: sent: 3191, lost: 0, retx: 0, rcvd: 586; batch: count: 342; min: 1; max: 15; avg: 6.26
10:17:45.990 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 470; cwnd: 86020; conn flow: max: 27695042, avail: 1504711; packets: sent: 2393, lost: 3, retx: 3, rcvd: 462; batch: count: 272; min: 1; max: 15; avg: 6.02
10:17:46.984 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 606; cwnd: 86020; conn flow: max: 31647905, avail: 867216; packets: sent: 3237, lost: 2, retx: 2, rcvd: 601; batch: count: 344; min: 1; max: 16; avg: 6.94
10:17:47.034 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 549; cwnd: 86020; conn flow: max: 36394296, avail: 1286083; packets: sent: 3048, lost: 1, retx: 1, rcvd: 556; batch: count: 318; min: 1; max: 15; avg: 6.14
10:17:48.045 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 472; cwnd: 86020; conn flow: max: 39549202, avail: 925447; packets: sent: 2477, lost: 2, retx: 2, rcvd: 471; batch: count: 270; min: 1; max: 15; avg: 11.07
10:17:49.090 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 533; cwnd: 86020; conn flow: max: 44291453, avail: 1483411; packets: sent: 2948, lost: 1, retx: 1, rcvd: 541; batch: count: 329; min: 1; max: 15; avg: 6.11
10:17:50.153 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 481; cwnd: 86020; conn flow: max: 47450524, avail: 980767; packets: sent: 2581, lost: 1, retx: 1, rcvd: 480; batch: count: 287; min: 1; max: 15; avg: 5.11
10:17:51.235 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 446; cwnd: 86020; conn flow: max: 50618119, avail: 780658; packets: sent: 2380, lost: 7, retx: 7, rcvd: 453; batch: count: 264; min: 1; max: 15; avg: 6.09
10:17:52.265 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 601; cwnd: 86020; conn flow: max: 55361702, avail: 933970; packets: sent: 3237, lost: 2, retx: 2, rcvd: 598; batch: count: 337; min: 1; max: 15; avg: 6.89
10:17:53.266 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 531; cwnd: 86020; conn flow: max: 59320195, avail: 813214; packets: sent: 2874, lost: 1, retx: 1, rcvd: 533; batch: count: 310; min: 1; max: 15; avg: 9.26
10:17:54.337 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 491; cwnd: 86020; conn flow: max: 63277274, avail: 1057449; packets: sent: 2614, lost: 0, retx: 0, rcvd: 495; batch: count: 286; min: 1; max: 14; avg: 10.26
10:17:55.412 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 536; cwnd: 86020; conn flow: max: 67240041, avail: 835997; packets: sent: 2950, lost: 3, retx: 3, rcvd: 546; batch: count: 314; min: 1; max: 15; avg: 6.28
10:17:56.459 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 549; cwnd: 86020; conn flow: max: 71987844, avail: 1487671; packets: sent: 2887, lost: 1, retx: 1, rcvd: 539; batch: count: 315; min: 1; max: 15; avg: 5.56
10:17:57.555 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 510; cwnd: 86020; conn flow: max: 75936368, avail: 1514651; packets: sent: 2763, lost: 1, retx: 1, rcvd: 510; batch: count: 304; min: 1; max: 15; avg: 6.38
10:17:58.605 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 609; cwnd: 86020; conn flow: max: 79876321, avail: 773460; packets: sent: 3298, lost: 1, retx: 1, rcvd: 605; batch: count: 343; min: 1; max: 15; avg: 10.11
10:17:59.646 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 605; cwnd: 86020; conn flow: max: 84608751, avail: 925435; packets: sent: 3228, lost: 1, retx: 1, rcvd: 597; batch: count: 345; min: 1; max: 14; avg: 7.00
10:18:00.685 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 499; cwnd: 86020; conn flow: max: 88543079, avail: 1006319; packets: sent: 2715, lost: 1, retx: 1, rcvd: 511; batch: count: 298; min: 1; max: 15; avg: 10.43
10:18:01.754 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 561; cwnd: 86020; conn flow: max: 93281118, avail: 1366986; packets: sent: 3082, lost: 1, retx: 1, rcvd: 575; batch: count: 344; min: 1; max: 15; avg: 8.29
10:18:02.804 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 512; cwnd: 86020; conn flow: max: 97246747, avail: 1425199; packets: sent: 2754, lost: 0, retx: 0, rcvd: 520; batch: count: 304; min: 1; max: 15; avg: 5.30
10:18:03.796 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 537; cwnd: 86020; conn flow: max: 101210909, avail: 1269030; packets: sent: 2905, lost: 3, retx: 3, rcvd: 547; batch: count: 318; min: 1; max: 15; avg: 6.88
10:18:04.805 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 598; cwnd: 86020; conn flow: max: 105951844, avail: 1381187; packets: sent: 3264, lost: 3, retx: 3, rcvd: 592; batch: count: 349; min: 1; max: 15; avg: 6.69
10:18:05.827 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 682; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 776, lost: 3, retx: 3, rcvd: 6418; batch: count: 581; min: 1; max: 14; avg: 1.00
10:18:06.922 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 592; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 519, lost: 2, retx: 2, rcvd: 5920; batch: count: 518; min: 1; max: 2; avg: 1.00
10:18:07.985 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 570; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 498, lost: 4, retx: 4, rcvd: 5845; batch: count: 493; min: 1; max: 2; avg: 1.00
10:18:08.057 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 782; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 678, lost: 3, retx: 3, rcvd: 6938; batch: count: 675; min: 1; max: 2; avg: 1.00
10:18:09.126 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 550; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 485, lost: 5, retx: 5, rcvd: 5806; batch: count: 480; min: 1; max: 2; avg: 1.05
10:18:10.195 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 504; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 443, lost: 4, retx: 4, rcvd: 5444; batch: count: 439; min: 1; max: 2; avg: 1.00
10:18:11.265 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 702; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 607, lost: 4, retx: 4, rcvd: 6878; batch: count: 603; min: 1; max: 2; avg: 1.00
10:18:12.287 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 597; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 520, lost: 6, retx: 6, rcvd: 6007; batch: count: 514; min: 1; max: 2; avg: 1.00
10:18:13.285 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 539; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 469, lost: 3, retx: 3, rcvd: 5634; batch: count: 466; min: 1; max: 2; avg: 1.00
10:18:14.284 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 592; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 516, lost: 2, retx: 2, rcvd: 5989; batch: count: 514; min: 1; max: 2; avg: 1.00
10:18:15.354 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 642; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 560, lost: 3, retx: 3, rcvd: 6305; batch: count: 557; min: 1; max: 2; avg: 1.00
10:18:16.377 [NOTICE] conn-stats: DFE672238E9D8E52: ticks: 662; cwnd: 86020; conn flow: max: 105951844, avail: 1094236; packets: sent: 570, lost: 4, retx: 4, rcvd: 6273; batch: count: 566; min: 1; max: 2; avg: 1.00
10:18:16.163 [NOTICE] conn-stats: busy connection DFE672238E9D8E52 is destroyed
10:18:16.169 [NOTICE] [QUIC:DFE672238E9D8E52] sendctl: stats: n_total_sent: 80383; n_resent: 0; n_delayed: 0
10:18:16.179 [NOTICE] Connection closed
10:18:16.189 [NOTICE] [QUIC:DFE672238E9D8E52] conn: # ticks: 20761
10:18:16.205 [NOTICE] [QUIC:DFE672238E9D8E52] conn: sent 80383 packets
10:18:16.205 [NOTICE] [QUIC:DFE672238E9D8E52] conn: received 87376 packets, of which 0 were not decryptable, 0 were dups and 0 were errors; sent 80383 packets, avg stream data per outgoing packet is 1304 bytes
10:18:16.205 [NOTICE] [QUIC:DFE672238E9D8E52] conn: delayed ACKs settings: (3/1.000/0.800/0.350/0.050/0.050); packet tolerances sent: count: 0, min: 0, max: 0
10:18:16.215 [NOTICE] [QUIC:DFE672238E9D8E52] conn: ACKs: delayed acks on: yes; in: 13803; processed: 12291; merged: 1512

This is abysmal performance on Windows and doesn't even scratch significant CPU usage.

On Linux, it not only doesn't peg the CPU, it has both bimodal metrics (eg. every other run is twice the bandwidth) and doesn't appear to detect the correct MTU.

> perf_client -p 104857600:104857600 -s ::1:443 -H localhost -L notice -z 12000
10:22:51.997 [NOTICE] conn-stats: connection E96B936EBDC063BA marked busy: it's had 16 immediate ticks in a row
10:22:51.997 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 186; cwnd: 64518; conn flow: max: 11275393, avail: 1382159; packets: sent: 829, lost: 2, retx: 2, rcvd: 189; batch: count: 179; min: 1; max: 6; avg: 1.73
10:22:52.987 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 1639; cwnd: 67710; conn flow: max: 102628343, avail: 857678; packets: sent: 7717, lost: 9, retx: 9, rcvd: 1609; batch: count: 1484; min: 1; max: 8; avg: 6.00
10:22:53.114 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 702; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 841, lost: 5, retx: 5, rcvd: 6272; batch: count: 617; min: 1; max: 7; avg: 1.00
10:22:54.183 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 565; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 493, lost: 3, retx: 3, rcvd: 4908; batch: count: 490; min: 1; max: 2; avg: 1.00
10:22:55.301 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 569; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 499, lost: 5, retx: 5, rcvd: 5097; batch: count: 494; min: 1; max: 2; avg: 1.00
10:22:56.353 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 542; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 466, lost: 2, retx: 2, rcvd: 4796; batch: count: 465; min: 1; max: 2; avg: 1.00
10:22:57.461 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 539; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 471, lost: 5, retx: 5, rcvd: 4836; batch: count: 465; min: 1; max: 2; avg: 1.00
10:22:58.538 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 534; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 462, lost: 2, retx: 2, rcvd: 4820; batch: count: 460; min: 1; max: 2; avg: 1.00
10:22:59.558 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 547; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 476, lost: 5, retx: 5, rcvd: 4783; batch: count: 471; min: 1; max: 2; avg: 1.00
10:23:00.609 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 566; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 497, lost: 3, retx: 3, rcvd: 4991; batch: count: 494; min: 1; max: 2; avg: 1.00
10:23:01.677 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 541; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 471, lost: 0, retx: 0, rcvd: 4818; batch: count: 471; min: 1; max: 1; avg: 1.00
10:23:02.806 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 565; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 491, lost: 4, retx: 4, rcvd: 4922; batch: count: 487; min: 1; max: 2; avg: 1.05
10:23:03.892 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 547; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 477, lost: 4, retx: 4, rcvd: 4999; batch: count: 473; min: 1; max: 2; avg: 1.00
10:23:04.880 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 536; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 466, lost: 3, retx: 3, rcvd: 4792; batch: count: 463; min: 1; max: 2; avg: 1.00
10:23:05.968 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 472; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 409, lost: 3, retx: 3, rcvd: 4126; batch: count: 406; min: 1; max: 2; avg: 1.02
10:23:06.149 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 540; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 464, lost: 3, retx: 3, rcvd: 4771; batch: count: 461; min: 1; max: 2; avg: 1.00
10:23:07.201 [NOTICE] conn-stats: E96B936EBDC063BA: ticks: 559; cwnd: 72714; conn flow: max: 105870518, avail: 1012910; packets: sent: 483, lost: 2, retx: 2, rcvd: 4909; batch: count: 481; min: 1; max: 2; avg: 1.00
10:23:07.510 [NOTICE] conn-stats: busy connection E96B936EBDC063BA is destroyed
10:23:07.510 [NOTICE] [QUIC:E96B936EBDC063BA] sendctl: stats: n_total_sent: 16025; n_resent: 0; n_delayed: 0
10:23:07.510 [NOTICE] Connection closed
10:23:07.510 [NOTICE] [QUIC:E96B936EBDC063BA] conn: # ticks: 10162
10:23:07.510 [NOTICE] [QUIC:E96B936EBDC063BA] conn: sent 16025 packets
10:23:07.510 [NOTICE] [QUIC:E96B936EBDC063BA] conn: received 75763 packets, of which 0 were not decryptable, 0 were dups and 0 were errors; sent 16025 packets, avg stream data per outgoing packet is 6543 bytes
10:23:07.510 [NOTICE] [QUIC:E96B936EBDC063BA] conn: delayed ACKs settings: (3/1.000/0.800/0.350/0.050/0.050); packet tolerances sent: count: 0, min: 0, max: 0
10:23:07.510 [NOTICE] [QUIC:E96B936EBDC063BA] conn: ACKs: delayed acks on: yes; in: 2340; processed: 2339; merged: 1

Still absymal performance.

On Linux, setting -z 8000 gives a much higher performance.

What is going on here?

Why is there any loss going to localhost on windows and linux?

Rjvs commented 2 years ago

We have tested the following scenarios and not managed to peg the CPU or get reasonable performance:

lsquic.dll integrated into our own application under Windows localhost; 30Mbit/s
perf_client/perf_server on Windows bare meta localhostl; 10Mbit/s
perf_client/perf_server on Windows tuning packet size; 65Mbit/s
perf_client/perf_server on Linux VM localhost; 30Mbit/s
perf_client/perf_server on Linux VM tuning packet size etc; 80Mbit/s

Is this the sort of performance that we should be expecting?

Wizmann commented 2 years ago

I'm not currently working on lsquic. But I guess we may check the buffer and pacing parameters, they can both highly affect on performance.

litespeedtech / lsquic

what is the maximum bitrate lsquic can reach? #123