How to reproduce the throughput shown in Figure 12 of your paper?

csl-ajou / DeepPlan

Fast and Efficient Model Serving Using Multi-GPUs with Direct-Host-Access (ACM EuroSys '23)

MIT License

50 stars 8 forks source link

How to reproduce the throughput shown in Figure 12 of your paper? #6

Open chenglimin opened 1 week ago

chenglimin commented 1 week ago

The method described in the README of this repo for reproducing Figure 12 does not actually reproduce the throughput. How to reproduce the throughput result?

JinuJeong commented 1 week ago

Since our experiments focus on tail latency and goodput, we do not measure throughput for the experiment. Some code modifications will be necessary to obtain throughput results. If you need the result, I can update the code for you.

JinuJeong commented 1 week ago

Is the Figure 12 that you mentioned the batching experiment? The order of figures changed after the camera-ready version. The Figure 12 described in the repository actually refers to Figure 13 in the final paper. In the batching experiment, throughputs were calculated using the formula batch_size / latency.