search

Xilinx / inference-server

https://xilinx.github.io/inference-server/
Apache License 2.0
43 stars 13 forks source link

Add performance graphs #198

Closed varunsh-xilinx closed 1 year ago

varunsh-xilinx commented 1 year ago

Summary of Changes

Motivation

After #197, we want to benchmark the inference server and track changes over time.

Implementation

I added scripts to create graphs from the gathered benchmark data and show them on the website with sphinx-charts under a new page called Performance.

Notes

In the future, running similar kinds of benchmarks could be part of the CI to ensure that changes don't degrade the server performance. However, some open questions need to be answered first:

  1. How long should the test run for to be meaningful? In initial tests, even at the MLPerf limits, there were variations in the results.
  2. Temporary dips in performance may be tolerable e.g. if the cause is known. It shouldn't immediately reject the PR if the performance drops.
  3. How much of a drop is significant enough to merit remedial action?
gbuildx commented 1 year ago

Build successful!