It allows to be asynchronous when processing requests, and it is build-in with FastAPI.
I made a small test with ab on one worker thread and 4 concurrent clients and it is about 30% faster :
bench async
$ ab -p boaviztapi.json -T application/json -n 1000 -c 4 localhost:8000/v1/component/cpu
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Finished 1000 requests
Server Software: uvicorn
Server Hostname: localhost
Server Port: 8000
Document Path: /v1/component/cpu
Document Length: 374 bytes
Concurrency Level: 4
Time taken for tests: 1.260 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 519000 bytes
Total body sent: 198000
HTML transferred: 374000 bytes
Requests per second: 793.92 [#/sec] (mean)
Time per request: 5.038 [ms] (mean)
Time per request: 1.260 [ms] (mean, across all concurrent requests)
Transfer rate: 402.39 [Kbytes/sec] received
153.51 kb/s sent
555.90 kb/s total
bench sync
$ ab -p boaviztapi.json -T application/json -n 1000 -c 4 localhost:8000/v1/component/cpu
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Finished 1000 requests
Server Software: uvicorn
Server Hostname: localhost
Server Port: 8000
Document Path: /v1/component/cpu
Document Length: 374 bytes
Concurrency Level: 4
Time taken for tests: 1.934 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 519000 bytes
Total body sent: 198000
HTML transferred: 374000 bytes
Requests per second: 517.09 [#/sec] (mean)
Time per request: 7.736 [ms] (mean)
Time per request: 1.934 [ms] (mean, across all concurrent requests)
Transfer rate: 262.08 [Kbytes/sec] received
99.98 kb/s sent
362.06 kb/s total
The file boaviztapi.json was containing :
{"core_units": 12, "die_size_per_core": 0.245}
for the server test I used the data from test_dell_r740_server. The performance improvement is not as good as the previous one (~2,5%) but if we go async it is better to handle all io operations with async.
Use asyncio for HTTP API.
It allows to be asynchronous when processing requests, and it is build-in with FastAPI.
I made a small test with ab on one worker thread and 4 concurrent clients and it is about 30% faster :
bench async
bench sync
The file
boaviztapi.json
was containing :for the server test I used the data from
test_dell_r740_server
. The performance improvement is not as good as the previous one (~2,5%) but if we go async it is better to handle all io operations with async.