Closed technicianted closed 1 year ago
Output timestamp for human format and include it as a json field for jsonl:
2023-10-31 01:00:54 rpm: 5.0 requests: 5 failures: 0 throttled: 0 ctx_tpm: 2500.0 gen_tpm: 515.0 ttft_avg: 0.82 ttft_95th: 1.296 tbt_avg: 0.047 tbt_95th: 0.048 e2e_avg: 5.619 e2e_95th: 6.212 util_avg: 99.4% util_95th: 99.9%
{"run_seconds": 6, "timestamp": "2023-10-31 01:06:08", "rpm": 4.0, "requests": 4, "failures": 0, "throttled": 0, "tpm": {"context": 2004.0, "gen": 412.0}, "e2e": {"avg": 3.971, "95th": 4.274}, "ttft": {"avg": 1.181, "95th": 1.221}, "tbt": {"avg": 0.027, "95th": 0.03}, "util": {"avg": "57.6%", "95th": "58.7%"}}
fixes #4
Output timestamp for human format and include it as a json field for jsonl:
fixes #4