search

UChi-JCL / CacheGen

54 stars 8 forks source link

how to calculate TTFT ? #3

Open liyanboSustech opened 1 month ago

liyanboSustech commented 1 month ago

我想问一下TTFT是怎么计算出来的啊？有没有TTFT的计算脚本，谢谢~

Hanchenli commented 3 weeks ago

An easy way to calculate TTFT is to limit the maximum generation length of LLM to be 1 and take the time difference between the start and the generation finish time.

liyanboSustech commented 3 weeks ago

Really appricate your timely reply. Enjoy your day!😊