Hi! I noticed that the output of the examples under taso/examples folder provided cost of the graph, but cost is not exactly the end-to-end runtime.
I tried to output the end-to-end runtime of the resnet50 model before and after optimization by looking at the code in taso/examples/eval_joint.py and taso/examples/resnet50.py, and this is what I got:
import taso as ts
import onnx
def resnet_block(graph, input, strides, out_channels):
w1 = graph.new_weight(dims=(out_channels,input.dim(1),1,1))
t = graph.conv2d(input=input, weight=w1,
strides=(1,1), padding="SAME",
activation="RELU")
w2 = graph.new_weight(dims=(out_channels,t.dim(1),3,3))
t = graph.conv2d(input=t, weight=w2,
strides=strides, padding="SAME",
activation="RELU")
w3 = graph.new_weight(dims=(4*out_channels,t.dim(1),1,1))
t = graph.conv2d(input=t, weight=w3,
strides=(1,1), padding="SAME")
if (strides[0]>1) or (input.dim(1) != out_channels*4):
w4 = graph.new_weight(dims=(out_channels*4,input.dim(1),1,1))
input=graph.conv2d(input=input, weight=w4,
strides=strides, padding="SAME",
activation="RELU")
return graph.relu(graph.add(input, t))
graph = ts.new_graph()
input = graph.new_input(dims=(1,3,224,224)) #1,64,56,56))
t = input
for i in range(3):
t = resnet_block(graph, t, (1,1), 64)
strides = (2,2)
for i in range(4):
t = resnet_block(graph, t, strides, 128)
strides = (1,1)
strides = (2,2)
for i in range(6):
t = resnet_block(graph, t, strides, 256)
strides = (1,1)
strides = (2,2)
for i in range(3):
t = resnet_block(graph, t, strides, 512)
strides = (1,1)
orig_onnx_model = ts.export_onnx(graph)
onnx.save(orig_onnx_model, "resnet50_orig.onnx")
print("Original graph runtime:")
print("TASO: end-to-end inference time = {}ms".format(graph.run_time()))
new_graph = ts.optimize(graph, alpha=1.0, budget=1000)
onnx_model = ts.export_onnx(new_graph)
onnx.save(onnx_model, "resnet50_optimized.onnx")
print("Optimized graph runtime:")
print("TASO: end-to-end inference time = {}ms".format(new_graph.run_time()))
However, when I ran the above code, I encountered errors when getting the runtime of the optimized model:
Hi! I noticed that the output of the examples under taso/examples folder provided cost of the graph, but cost is not exactly the end-to-end runtime.
I tried to output the end-to-end runtime of the resnet50 model before and after optimization by looking at the code in taso/examples/eval_joint.py and taso/examples/resnet50.py, and this is what I got:
However, when I ran the above code, I encountered errors when getting the runtime of the optimized model:
Do you know what might be the problem? Or is there a better way to get the end-to-end runtime? Thank you very much for your time!