Open Apisteftos opened 1 year ago
It has nothing to do with the size of the json file. It's the # of events. See https://github.com/microsoft/onnxruntime/blob/cf19c3697d126982acf45c040729269e53b5d9f4/onnxruntime/core/common/profiler.cc#L10
Why in server is 2.2GB ? The number of the events is the same on my PC and server. I suppose it has with the inference time?
#else
profiling::Profiler::~Profiler() {}
#endif
::onnxruntime::TimePoint profiling::Profiler::Start() {
ORT_ENFORCE(enabled_);
auto start_time = std::chrono::high_resolution_clock::now();
auto ts = TimeDiffMicroSeconds(profiling_start_time_, start_time);
for (const auto& ep_profiler : ep_profilers_) {
ep_profiler->Start(ts);
}
return start_time;
}
How to change the settings if there are provided?
Do you really need so many events? If you profile to investigate perf issue, you may try reducing the repeat count.
Describe the issue
I am doing inference with my personal PC with batch_size 1 and profiling file reaches almost 483MB. Has any idea why is that happening? How can I increase the size of the profiled json file?
To reproduce
Profiling with DenseNet121 from onnx zoo on GPU. The exctracted json file reaches 483 and the stops to profile. An error occures:
I tried the same on a server and it reaches 2.2GB why this difference? I suppose it has with the SSD formating type?
Urgency
No response
Platform
Linux
OS Version
Ubuntu 22.04
ONNX Runtime Installation
Released Package
ONNX Runtime Version or Commit ID
1.15.0
ONNX Runtime API
Python
Architecture
X64
Execution Provider
CUDA
Execution Provider Library Version
CUDA 11.6