Enable ITL, TTFT, E2E latency computation using mean rather than median

fmperf-project / fmperf

Cloud Native Benchmarking of Foundation Models

Apache License 2.0

21 stars 10 forks source link

Open tdoublep opened 5 months ago

tdoublep commented 5 months ago

Let's have it configurable in the parser, and even maybe make mean the default.

Median does not make sense with speculative decoding anyway.