search

microsoft / onnxruntime

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator
https://onnxruntime.ai
MIT License
14.59k stars 2.92k forks source link

`CleanUnusedInitializersAndNodeArgs` warnings are printed only with subgraphs #14694

Open fxmarty opened 1 year ago

fxmarty commented 1 year ago

Describe the issue

When using a model with subgraphs, [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_11_output_0'. It is not used by any node and should be removed from the model. warnings are printed by ONNX Runtime.

However, using the very same model without subgraphs, no warnings are printed.

To reproduce

Check the models in: https://huggingface.co/fxmarty/CleanUnusedInitializersAndNodeArgs-prints-ort/tree/main

Netron of decoder_model.onnx: https://netron.app/?url=https://huggingface.co/fxmarty/CleanUnusedInitializersAndNodeArgs-prints-ort/blob/main/decoder_model.onnx Netron of decoder_with_past_model.onnx: https://netron.app/?url=https://huggingface.co/fxmarty/CleanUnusedInitializersAndNodeArgs-prints-ort/blob/main/decoder_with_past_model.onnx Netron of decoder_model_merged.onnx (that has an If node that dispatches on the two previous ONNX subgraphs): https://netron.app/?url=https://huggingface.co/fxmarty/CleanUnusedInitializersAndNodeArgs-prints-ort/blob/main/decoder_model_merged.onnx

When running inference with decoder_model.onnx or decoder_with_past_model.onnx with ort.GraphOptimizationLevel>=ORT_ENABLE_BASIC, no CleanUnusedInitializersAndNodeArgs warning is printed. However, using the merged model (that basically has decoder_model.onnx and decoder_with_past_model.onnx as subgraphs of an If node), tons of CleanUnusedInitializersAndNodeArgs are printed.

Why is there a difference when using subgraphs?

Truncated `CleanUnusedInitializersAndNodeArgs` log ``` 2023-02-15 14:49:00.094009769 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_18_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.094040264 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_18_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.094048836 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_18_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.094064775 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_18_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.094088472 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_18_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.094251747 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_18_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.094270753 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_18_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.094279229 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_18_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.094294351 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_18_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.094299195 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_18_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101935573 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_9_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101946610 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_6_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101950921 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_4_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101954159 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_755'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101959415 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1138'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101962744 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_5_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101980096 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_1_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101984460 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101987324 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_590'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101990261 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_588'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101993033 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_36_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101995852 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_35_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.101998649 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_789'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102001704 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_11_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102008788 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_7_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102012231 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_12_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102015157 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_11_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102018029 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_12_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102020936 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_516'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102023731 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_8_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102026751 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_499'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102029612 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_6_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102032560 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_5_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102035605 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_4_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102038730 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_6_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102041493 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_2_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102044410 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_497'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102047306 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_514'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102050584 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_2_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102053690 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_11_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102056611 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_8_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102059633 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_205'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102062488 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_10_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102068063 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_3_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102071043 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1049'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102073946 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1305'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102076799 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_237'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102079718 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_4_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102082742 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_12_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102086102 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_791'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102089461 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_7_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102092288 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_7_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102095292 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_2_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102099161 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_36_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102102682 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_7_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102106724 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_5_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102110069 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_8_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102113810 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_222'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102117570 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_774'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102121492 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_35_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102127514 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1066'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102133414 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_757'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102137394 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_239'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102141149 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_3_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102144693 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_312'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102148670 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_9_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102152887 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_9_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102157140 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_772'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102160908 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_9_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102166614 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_96'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102170316 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_36_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102173632 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_3_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102177383 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102181715 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1415'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102185432 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_2_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102189552 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_1_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102193427 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1322'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102197366 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_10_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102200987 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_10_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102205447 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_480'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102210847 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_10_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102214847 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_1_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102218346 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_11_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102222349 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_863'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102226218 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_865'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102229556 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102233781 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_5_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102237533 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_9_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102241162 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_1_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102245253 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_36_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102249219 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1064'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102252687 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_314'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102256738 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/Constant_2_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102261449 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1307'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102266056 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_1_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102269916 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1030'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102273622 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_6_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102278658 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1047'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102284307 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_7_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102292023 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_8_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102295138 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_11_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102298433 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_220'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102302858 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_8_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102307073 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1413'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102311314 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_35_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102315659 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_36_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102319321 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_35_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102323385 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_482'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102328816 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_6_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102333381 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_203'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102337651 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_2_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102341713 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_3_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102345824 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_12_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102350057 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_3_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102353598 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_4_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102357803 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_5_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102361477 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1324'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102366915 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1339'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102370552 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1341'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102374581 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_12_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102379425 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102383616 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_4_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102388852 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102392527 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_10_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102396372 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1140'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102400075 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_35_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102403768 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1032'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102628953 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_9_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102636149 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_752'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102640453 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_6_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102644573 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_4_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102648335 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_733'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102651972 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_1_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102657125 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1112'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102660811 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_5_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102665486 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_37_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102671860 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_566'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102675338 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_36_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102679148 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_35_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102682722 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_767'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102686771 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.0/attn/Constant_4_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102690315 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1296'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102694058 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.2/attn/Constant_11_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102697612 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_1298'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102701544 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.4/attn/Constant_7_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102705460 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_841'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102709307 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_8_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102713090 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_750'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102716869 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_12_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102720855 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.3/attn/Constant_7_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102724875 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer '/transformer/h.1/attn/Constant_11_output_0'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102728921 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_496'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102732668 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_494'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102736553 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_479'. It is not used by any node and should be removed from the model. 2023-02-15 14:49:00.102742175 [W:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'onnx::Unsqueeze_462'. It is not used by any node and should be removed from the model. ... ```

Urgency

Medium

Platform

Linux

OS Version

Linux 5.15.0-56-generic #62-Ubuntu SMP Tue Nov 22 19:54:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

ONNX Runtime Installation

Released Package

ONNX Runtime Version or Commit ID

1.14.0

ONNX Runtime API

Python

Architecture

X64

Execution Provider

Default CPU

Execution Provider Library Version

No response

tianleiwu commented 1 year ago

The transformer optimizer script has limitations. For example prune graph does not work when there is subgraph: see here.

A walkaround is to optimize decoder model and decoder with past state separately. Example of export and optimize two models separately: https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/models/t5/convert_to_onnx.py

Then combine them into subgraphs (and also consolidate weights since two subgraphs share some weights) . An example of such approach is a script that converts GPT2 or T5 to use beam search: https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/python/tools/transformers/convert_generation.py

fxmarty commented 1 year ago

@tianleiwu Thanks a lot for your answer! So if I understand correctly, an optimization that is normally done when ort.GraphOptimizationLevel>=ORT_ENABLE_BASIC is not performed when having subgraphs, and thus the CleanUnusedInitializersAndNodeArgs warnings are triggered at a later stage only when the ONNX has subgraphs?

tianleiwu commented 1 year ago

@fxmarty,

You can try load the model and save optimized model from onnxruntime. When you create session option, there is a field to export optimized model path. Example code: https://github.com/microsoft/onnxruntime/blob/8372c86e7f15ed0d8d47e1e0afc115c30db39252/onnxruntime/python/tools/transformers/optimizer.py#L104

I think that shall be able to remove unused initializer.

To fix the root cause, that need update the transformers optimizer script to handle subgraph nicely. We can add this to our backlog.

veriff-javiervgd commented 1 year ago

This warning has to do with graph level optimization in the onnxruntime session. ONNX has 3 levels of graph optimization: Basic, Extended and Layout Optimizations. You can define your optimization level with the following enums:

onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL
onnxruntime.GraphOptimizationLevel.ORT_ENABLE_BASIC
onnxruntime.GraphOptimizationLevel.ORT_ENABLE_EXTENDED
onnxruntime.GraphOptimizationLevel.ORT_ENABLE_ALL

If you disable graph optimizations, ORT_DISABLE_ALL, in the session's options then you will remove those warnings:

# Session options
sess_options = onnxruntime.SessionOptions()
sess_options.graph_optimization_level = onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL

# ONNX session setup
session = onnxruntime.InferenceSession(
    "model.onnx", 
    providers=["CPUExecutionProvider"], 
    sess_options=sess_options
)

Another way is to use the default graph optimization and just to increase the severity of the logger to filter out noisy warnings:

# Session options
sess_options = onnxruntime.SessionOptions()
sess_options.log_severity_level = 3

# ONNX session setup
session = onnxruntime.InferenceSession(
    "model.onnx", 
    providers=["CPUExecutionProvider"], 
    sess_options=sess_options
)