Open Shadow-Devil opened 1 month ago
Description: I'm currently adding Ballerina to the FrameworkBenchmark and found a bug while implementing the plaintext test.
Here is my primitive implementation of this test:
import ballerina/http; service / on new http:Listener(8080) { # Test 6 resource function get plaintext() returns string { return "Hello, World!"; } }
When running this locally I get the following log messages:
BENCHMARKING PLAINTEXT ... --------------------------------------------------------- Running Primer plaintext wrk -H 'Host: tfb-server' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 5 -c 8 --timeout 8 -t 8 http://tfb-server:8080/plaintext --------------------------------------------------------- Running 5s test @ http://tfb-server:8080/plaintext 8 threads and 8 connections Thread Stats Avg Stdev Max +/- Stdev Latency 8.03ms 14.33ms 194.22ms 92.25% Req/Sec 239.74 158.85 595.00 56.93% Latency Distribution 50% 3.16ms 75% 8.44ms 90% 18.62ms 99% 74.21ms 9540 requests in 5.03s, 1.21MB read Requests/sec: 1894.78 Transfer/sec: 246.10KB --------------------------------------------------------- Running Warmup plaintext wrk -H 'Host: tfb-server' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 512 --timeout 8 -t 8 http://tfb-server:8080/plaintext --------------------------------------------------------- Running 15s test @ http://tfb-server:8080/plaintext 8 threads and 512 connections Thread Stats Avg Stdev Max +/- Stdev Latency 102.43ms 441.35ms 6.65s 94.83% Req/Sec 1.55k 2.08k 10.89k 86.88% Latency Distribution 50% 525.00us 75% 2.92ms 90% 132.67ms 99% 2.38s 100873 requests in 15.10s, 12.79MB read Requests/sec: 6680.91 Transfer/sec: 867.74KB --------------------------------------------------------- 8 threads and 256 connections Thread Stats Avg Stdev Max +/- Stdev Latency 0.00us 0.00us 0.00us -nan% Req/Sec 0.00 0.00 0.00 -nan% Latency Distribution 50% 0.00us 75% 0.00us 90% 0.00us 99% 0.00us 0 requests in 15.10s, 0.00B read Socket errors: connect 0, read 159208, write 0, timeout 0 Requests/sec: 0.00 Transfer/sec: 0.00B STARTTIME 1722778055 ENDTIME 1722778070 --------------------------------------------------------- Concurrency: 1024 for plaintext wrk -H 'Host: tfb-server' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 1024 --timeout 8 -t 8 http://tfb-server:8080/plaintext -s pipeline.lua -- 16 --------------------------------------------------------- Running 15s test @ http://tfb-server:8080/plaintext 8 threads and 1024 connections Thread Stats Avg Stdev Max +/- Stdev Latency 0.00us 0.00us 0.00us -nan% Req/Sec 0.00 0.00 0.00 -nan% Latency Distribution 50% 0.00us 75% 0.00us 90% 0.00us 99% 0.00us 0 requests in 15.10s, 0.00B read Socket errors: connect 0, read 173555, write 0, timeout 0 Requests/sec: 0.00 Transfer/sec: 0.00B STARTTIME 1722778072 ENDTIME 1722778087 --------------------------------------------------------- Concurrency: 4096 for plaintext wrk -H 'Host: tfb-server' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 4096 --timeout 8 -t 8 http://tfb-server:8080/plaintext -s pipeline.lua -- 16 --------------------------------------------------------- Running 15s test @ http://tfb-server:8080/plaintext 8 threads and 4096 connections Thread Stats Avg Stdev Max +/- Stdev Latency 0.00us 0.00us 0.00us -nan% Req/Sec 0.00 0.00 0.00 -nan% Latency Distribution 50% 0.00us 75% 0.00us 90% 0.00us 99% 0.00us 0 requests in 15.09s, 0.00B read Socket errors: connect 0, read 74546, write 0, timeout 0 Requests/sec: 0.00 Transfer/sec: 0.00B STARTTIME 1722778089 ENDTIME 1722778104 --------------------------------------------------------- Concurrency: 16384 for plaintext wrk -H 'Host: tfb-server' -H 'Accept: text/plain,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 16384 --timeout 8 -t 8 http://tfb-server:8080/plaintext -s pipeline.lua -- 16 --------------------------------------------------------- Running 15s test @ http://tfb-server:8080/plaintext 8 threads and 16384 connections Thread Stats Avg Stdev Max +/- Stdev Latency 0.00us 0.00us 0.00us -nan% Req/Sec 0.00 0.00 0.00 -nan% Latency Distribution 50% 0.00us 75% 0.00us 90% 0.00us 99% 0.00us 0 requests in 15.09s, 0.00B read Socket errors: connect 0, read 16056, write 0, timeout 0 Requests/sec: 0.00 Transfer/sec: 0.00B STARTTIME 1722778106 ENDTIME 1722778122
After the initial "primer" and "warmup" phase there are no more requests processed.
You can see the full code here: https://github.com/Shadow-Devil/FrameworkBenchmarks/tree/master/frameworks/Ballerina/ballerina Run ./tfb --test ballerina --type plaintext to run the test case.
./tfb --test ballerina --type plaintext
Description: I'm currently adding Ballerina to the FrameworkBenchmark and found a bug while implementing the plaintext test.
Here is my primitive implementation of this test:
When running this locally I get the following log messages:
After the initial "primer" and "warmup" phase there are no more requests processed.
You can see the full code here: https://github.com/Shadow-Devil/FrameworkBenchmarks/tree/master/frameworks/Ballerina/ballerina Run
./tfb --test ballerina --type plaintextto run the test case.