How to measure the latency correctly?

mit-han-lab / once-for-all

[ICLR 2020] Once for All: Train One Network and Specialize it for Efficient Deployment

https://ofa.mit.edu/

MIT License

1.89k stars 333 forks source link

How to measure the latency correctly? #47

Closed tfwang08 closed 3 years ago

tfwang08 commented 3 years ago

Hi, Thanks for your great work! When I was testing the latency on V100, the results confused me. I used the following code to measure the latency table. torch.cuda.empty_cache() img_L = img_L.cuda() start.record() out = ofa_network(img_L) end.record() torch.cuda.synchronize() run_time.update(start.elapsed_time(end)) The img_L is one image. Is this correct?