search

microsoft / onnxconverter-common

Common utilities for ONNX converters
MIT License
240 stars 65 forks source link

Performance degrade after sess_options.enable_profiling = True #267

Open Jay19751103 opened 9 months ago

Jay19751103 commented 9 months ago

Hi @xiaowuhu

Using the tool referred in https://onnxruntime.ai/docs/performance/tune-performance/profiling-tools.html I measure stable diffusion with specify sess_options.enable_profiling = True before add it . I get 22.40it/s, after enabling it . I just can achieve 10.45. performance degrading from 22.40 to 10.45.

Could it be improved ?
Following is measured data

image

Another question is that I have two configurations, before enable profiling

config A is 22.40 it/s , config B is 20.23 it/s but after turn on the profiling with sess_options.enable_profiling = True config A is 10.45 it/s, config B is 11.03 it/s

Since the total number operations is different, I'm not sure it affect performance measuring or not config A is 56250 counts to measure config B is 49800 counts to measure.

xadupre commented 1 month ago

When you profile a model, you should exclude the first iteration when aggregating the metrics. This one slower as it tries to do some optimization (allocations, ...)