Benchmark scripts for TVM
Tested with
TVM commit id: 91e07e1f3a7 (Feb. 5, 2021)
mxnet==1.7.0
gluonnlp==0.10.0
AutoTVM
-------------------------------------------------------------
Network Name Batch size Mean Inference Time (std dev)
-------------------------------------------------------------
resnet_50 1 5.40 ms (0.08 ms)
mobilenet_v2 1 1.33 ms (0.05 ms)
bert 1 31.31 ms (0.11 ms)
-------------------------------------------------------------
AutoScheduler
-------------------------------------------------------------
Network Name Batch size Mean Inference Time (std dev)
-------------------------------------------------------------
resnet_50 1 5.30 ms (0.05 ms)
mobilenet_v2 1 0.91 ms (0.02 ms)
bert 1 16.52 ms (0.16 ms)
-------------------------------------------------------------
The following commands read pre-tuned logs from directory saved_logs/latest
and benchmark the latency for all networks.
Commands for AutoTVM
python3 benchmark_autotvm.py --network all --target "llvm -mcpu=skylake-avx512 -model=platinum-8124m" --logdir saved_logs/latest
Commands for AutoScheduler
python3 benchmark_autoscheduler.py --network all --target "llvm -mcpu=skylake-avx512 -model=platinum-8124m" --logdir saved_logs/latest
The following commands read pre-tuned logs from directory saved_logs/latest
and benchmark the latency for one network.
You can replace "resnet_50" below with "mobilenet_v2" or "bert".
Commands for AutoTVM
python3 benchmark_autotvm.py --network resnet_50 --target "llvm -mcpu=skylake-avx512 -model=platinum-8124m" --logdir saved_logs/latest
Commands for AutoScheduler
python3 benchmark_autoscheduler.py --network resnet_50 --target "llvm -mcpu=skylake-avx512 -model=platinum-8124m" --logdir saved_logs/latest
The following commands perform auto-tuning for one or all networks and save tuning logs to directory tmp_logs
.
After tuning, you can use these logs to run benchmark by using benchmark commands above and replace the last argument with --logdir tmp_logs
Commands for AutoTVM
# Tune one network
python3 tune_autotvm.py --network resnet_50 --target "llvm -mcpu=skylake-avx512 -model=platinum-8124m"
# Tune all networks
python3 tune_autotvm.py --network all --target "llvm -mcpu=skylake-avx512 -model=platinum-8124m"
Commands for AutoScheduler
# Tune one network
python3 tune_autoscheduler.py --network resnet_50 --target "llvm -mcpu=skylake-avx512 -model=platinum-8124m"
# Tune all networks
python3 tune_autoscheduler.py --network all --target "llvm -mcpu=skylake-avx512 -model=platinum-8124m"
AutoTVM
-------------------------------------------------------------
Network Name Batch size Mean Inference Time (std dev)
-------------------------------------------------------------
resnet_50 1 3.54 ms (0.02 ms)
mobilenet_v2 1 0.74 ms (0.00 ms)
bert 1 89.06 ms (1.22 ms)
-------------------------------------------------------------
AutoScheduler
-------------------------------------------------------------
Network Name Batch size Mean Inference Time (std dev)
-------------------------------------------------------------
resnet_50 1 2.90 ms (0.01 ms)
mobilenet_v2 1 0.57 ms (0.00 ms)
bert 1 9.95 ms (0.01 ms)
-------------------------------------------------------------
The following commands read pre-tuned logs from directory saved_logs/latest
and benchmark the latency for all networks.
Commands for AutoTVM
python3 benchmark_autotvm.py --network all --target "cuda -model=t4" --logdir saved_logs/latest
Commands for AutoScheduler
python3 benchmark_autoscheduler.py --network all --target "cuda -model=t4" --logdir saved_logs/latest
The following commands read pre-tuned logs from directory saved_logs/latest
and benchmark the latency for one network.
You can replace "resnet_50" below with "mobilenet_v2" or "bert".
Commands for AutoTVM
python3 benchmark_autotvm.py --network resnet_50 --target "cuda -model=t4" --logdir saved_logs/latest
Commands for AutoScheduler
python3 benchmark_autoscheduler.py --network resnet_50 --target "cuda -model=t4" --logdir saved_logs/latest
The following commands perform auto-tuning for one or all networks and save tuning logs to directory tmp_logs
.
After tuning, you can use these logs to run benchmark by using benchmark commands above and replace the last argument with --logdir tmp_logs
Commands for AutoTVM
# Tune one network
python3 tune_autotvm.py --network resnet_50 --target "cuda -model=t4"
# Tune all networks
python3 tune_autotvm.py --network all --target "cuda -model=t4"
Commands for AutoScheduler
# Tune one network
python3 tune_autoscheduler.py --network resnet_50 --target "cuda -model=t4"
# Tune all networks
python3 tune_autoscheduler.py --network all --target "cuda -model=t4"