Update the ResNet50 benchmark to work for all backends
Motivation
Having benchmarks to measure the server performance is important to find bugs and improve the performance.
Implementation
The updated benchmark has implementations for each backend to run a ResNet50 model. Each model is run with a variety of configurations and the results can be saved as JSON. I added a script to convert this JSON output into a format that can be placed in RST for the docs. I also found and added a way to add dynamic tables in the documentation which will be useful once we add more information.
The backend implementation can be generalized once another benchmark is added. It should be generic enough that we can reuse a lot of the logic but define unique preprocessing methods to support different models.
Notes
The benchmarks are now split by protocol. The reason for this is because HTTP (drogon) cannot be easily restarted within a process between different test runs. So each protocol is now in a separate executable which will start the server and the particular protocol once before running the benchmarks. This way, one protocol does not interfere with the others for benchmarks.
gbuildx
commented
1 year ago
Summary of Changes
Motivation
Having benchmarks to measure the server performance is important to find bugs and improve the performance.
Implementation
The updated benchmark has implementations for each backend to run a ResNet50 model. Each model is run with a variety of configurations and the results can be saved as JSON. I added a script to convert this JSON output into a format that can be placed in RST for the docs. I also found and added a way to add dynamic tables in the documentation which will be useful once we add more information.
The backend implementation can be generalized once another benchmark is added. It should be generic enough that we can reuse a lot of the logic but define unique preprocessing methods to support different models.
Notes
The benchmarks are now split by protocol. The reason for this is because HTTP (drogon) cannot be easily restarted within a process between different test runs. So each protocol is now in a separate executable which will start the server and the particular protocol once before running the benchmarks. This way, one protocol does not interfere with the others for benchmarks.