FMInference / FlexLLMGen

Running large language models on a single GPU for throughput-oriented scenarios.
Apache License 2.0
9.18k stars 548 forks source link

fix: use perf counter for benchmark timing #18

Closed kemingy closed 1 year ago

kemingy commented 1 year ago

Need to choose a monotonic clock for benchmark timing.

time.perf_counter should be much more accurate. I guess it's not necessary to use a perf_counter_ns.

Ref: https://docs.python.org/3/library/time.html#time.perf_counter