No CoreML backend???

[DeepFace-user]

I am doing a face model https://github.com/iperov/DeepFaceLive/releases/download/ZAHAR_LUPIN/Zahar_Lupin.dfm

environment: onnxruntime-coreml == 1.13.1 onnxruntime-silicon == 1.13.1

device : Apple Silicon M1

python: import onnx import onnxruntime as rt

options = rt.SessionOptions() options.log_severity_level = 0 options.intra_op_num_threads = 4 options.execution_mode = rt.ExecutionMode.ORT_SEQUENTIAL options.graph_optimization_level = rt.GraphOptimizationLevel.ORT_ENABLE_ALL

onnx

onnxSession = rt.InferenceSession(onnx_model_path, providers=[ {"CoreMLExecutionProvider"}], options)

logcat: EP Error using [{'CoreMLExecutionProvider'}] Falling back to ['CPUExecutionProvider'] and retrying. 2023-07-25 11:11:48.215511 [I:onnxruntime:, inference_session.cc:263 operator()] Flush-to-zero and denormal-as-zero are off 2023-07-25 11:11:48.215528 [I:onnxruntime:, inference_session.cc:271 ConstructorCommon] Creating and using per session threadpools since use_per_sessionthreads is true 2023-07-25 11:11:48.215535 [I:onnxruntime:, inference_session.cc:292 ConstructorCommon] Dynamic block base set to 0 EP Error using [{'CoreMLExecutionProvider'}] Falling back to ['CPUExecutionProvider'] and retrying. 2023-07-25 11:11:48.215793 [I:onnxruntime:, inference_session.cc:263 operator()] Flush-to-zero and denormal-as-zero are off 2023-07-25 11:11:48.215804 [I:onnxruntime:, inference_session.cc:271 ConstructorCommon] Creating and using per session threadpools since use_per_sessionthreads is true 2023-07-25 11:11:48.215810 [I:onnxruntime:, inference_session.cc:292 ConstructorCommon] Dynamic block base set to 0 EP Error using [{'CoreMLExecutionProvider'}] Falling back to ['CPUExecutionProvider'] and retrying. 2023-07-25 11:11:48.222091 [I:onnxruntime:, inference_session.cc:263 operator()] Flush-to-zero and denormal-as-zero are off 2023-07-25 11:11:48.222108 [I:onnxruntime:, inference_session.cc:271 ConstructorCommon] Creating and using per session threadpools since use_per_sessionthreads is true 2023-07-25 11:11:48.222116 [I:onnxruntime:, inference_session.cc:292 ConstructorCommon] Dynamic block base set to 0 2023-07-25 11:11:48.290081 [I:onnxruntime:, inference_session.cc:1222 Initialize] Initializing session. 2023-07-25 11:11:48.293215 [I:onnxruntime:, reshape_fusion.cc:42 ApplyImpl] Total fused reshape node count: 0 2023-07-25 11:11:48.295481 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295512 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295524 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295533 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295540 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295547 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295555 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295562 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295569 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295577 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295584 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295591 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295599 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295606 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295613 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295620 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295626 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.295634 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 11:11:48.296167 [V:onnxruntime:, session_state.cc:1010 VerifyEachNodeIsAssignedToAnEp] Node placements 2023-07-25 11:11:48.296189 [V:onnxruntime:, session_state.cc:1013 VerifyEachNodeIsAssignedToAnEp] All nodes placed on [CPUExecutionProvider]. Number of nodes: 69 2023-07-25 11:11:48.296278 [V:onnxruntime:, session_state.cc:66 CreateGraphInfo] SaveMLValueNameIndexMapping 2023-07-25 11:11:48.296307 [V:onnxruntime:, session_state.cc:112 CreateGraphInfo] Done saving OrtValue mappings. 2023-07-25 11:11:48.296561 [I:onnxruntime:, session_state_utils.cc:199 SaveInitializedTensors] Saving initialized tensors. 2023-07-25 11:11:48.297107 [I:onnxruntime:, session_state_utils.cc:342 SaveInitializedTensors] Done saving initialized tensors

############### The onruntime-silicon has been installed, but it only supports the CPU backend, not the CoreML backend, I don’t know the reason, can you help me.

Thanks in advance!

[cansik]

There should be no need to install multiple versions of onnxruntime. Either you install the onnxruntime-silicon or onnxruntime-coreml.

After correct installation, try to check what kind of providers are available:

import onnxruntime as rt
rt.get_available_providers()

[DeepFace-user]

@cansik Thank you rt.InferenceSession(onnx_model_or_path, providers= rt.get_available_providers(), sess_options=options)

Thank you very much, I have changed according to your form, but I am currently not sure whether to go CoreML, because the reasoning onnx is still very slow, no change

logcat:

2023-07-25 15:13:37.023599 [I:onnxruntime:, inference_session.cc:263 operator()] Flush-to-zero and denormal-as-zero are off 2023-07-25 15:13:37.023621 [I:onnxruntime:, inference_session.cc:271 ConstructorCommon] Creating and using per session threadpools since use_per_sessionthreads is true 2023-07-25 15:13:37.023630 [I:onnxruntime:, inference_session.cc:292 ConstructorCommon] Dynamic block base set to 0 2023-07-25 15:13:37.023842 [I:onnxruntime:, inference_session.cc:263 operator()] Flush-to-zero and denormal-as-zero are off 2023-07-25 15:13:37.023858 [I:onnxruntime:, inference_session.cc:271 ConstructorCommon] Creating and using per session threadpools since use_per_sessionthreads is true 2023-07-25 15:13:37.023865 [I:onnxruntime:, inference_session.cc:292 ConstructorCommon] Dynamic block base set to 0 2023-07-25 15:13:37.031563 [I:onnxruntime:, inference_session.cc:263 operator()] Flush-to-zero and denormal-as-zero are off 2023-07-25 15:13:37.031580 [I:onnxruntime:, inference_session.cc:271 ConstructorCommon] Creating and using per session threadpools since use_per_sessionthreads is true 2023-07-25 15:13:37.031588 [I:onnxruntime:, inference_session.cc:292 ConstructorCommon] Dynamic block base set to 0 2023-07-25 15:13:37.102138 [I:onnxruntime:, inference_session.cc:1222 Initialize] Initializing session. 2023-07-25 15:13:37.102163 [I:onnxruntime:, session_state.cc:31 SetupAllocators] Allocator already registered for OrtMemoryInfo:[name:Cpu id:0 OrtMemType:0 OrtAllocatorType:1 Device:[DeviceType:0 MemoryType:0 DeviceId:0]]. Ignoring allocator from CPUExecutionProvider 2023-07-25 15:13:37.103430 [I:onnxruntime:, reshape_fusion.cc:42 ApplyImpl] Total fused reshape node count: 0 2023-07-25 15:13:37.103527 [V:onnxruntime:, helper.cc:116 HasNeuralEngine] Current Apple hardware info: arm64 2023-07-25 15:13:37.103553 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Transpose] index: [0] name: [conv2d_150] supported: [1] 2023-07-25 15:13:37.103561 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [1] name: [conv2d_1] supported: [1] 2023-07-25 15:13:37.103567 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103572 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [2] name: [p_re_lu_1] supported: [0] 2023-07-25 15:13:37.103577 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [3] name: [depthwise_conv2d_1] supported: [1] 2023-07-25 15:13:37.103582 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [4] name: [conv2d_2] supported: [1] 2023-07-25 15:13:37.103587 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [5] name: [add_1] supported: [1] 2023-07-25 15:13:37.103592 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103597 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [6] name: [p_re_lu_2] supported: [0] 2023-07-25 15:13:37.103626 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [7] name: [depthwise_conv2d_2] supported: [1] 2023-07-25 15:13:37.103631 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [8] name: [conv2d_3] supported: [1] 2023-07-25 15:13:37.103635 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [9] name: [add_2] supported: [1] 2023-07-25 15:13:37.103640 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103646 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [10] name: [p_re_lu_3] supported: [0] 2023-07-25 15:13:37.103653 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [MaxPool] index: [11] name: [max_pooling2d_1] supported: [1] 2023-07-25 15:13:37.103657 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Pad] index: [12] name: [channel_padding_1] supported: [0] 2023-07-25 15:13:37.103662 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [13] name: [depthwise_conv2d_3] supported: [1] 2023-07-25 15:13:37.103667 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [14] name: [conv2d_4] supported: [1] 2023-07-25 15:13:37.103673 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [15] name: [add_3] supported: [1] 2023-07-25 15:13:37.103678 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103682 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [16] name: [p_re_lu_4] supported: [0] 2023-07-25 15:13:37.103687 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [17] name: [depthwise_conv2d_4] supported: [1] 2023-07-25 15:13:37.103692 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [18] name: [conv2d_5] supported: [1] 2023-07-25 15:13:37.103696 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [19] name: [add_4] supported: [1] 2023-07-25 15:13:37.103701 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103705 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [20] name: [p_re_lu_5] supported: [0] 2023-07-25 15:13:37.103710 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [21] name: [depthwise_conv2d_5] supported: [1] 2023-07-25 15:13:37.103714 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [22] name: [conv2d_6] supported: [1] 2023-07-25 15:13:37.103719 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [23] name: [add_5] supported: [1] 2023-07-25 15:13:37.103723 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103727 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [24] name: [p_re_lu_6] supported: [0] 2023-07-25 15:13:37.103733 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [MaxPool] index: [25] name: [max_pooling2d_2] supported: [1] 2023-07-25 15:13:37.103737 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Pad] index: [26] name: [channel_padding_2] supported: [0] 2023-07-25 15:13:37.103741 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [27] name: [depthwise_conv2d_6] supported: [1] 2023-07-25 15:13:37.103746 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [28] name: [conv2d_7] supported: [1] 2023-07-25 15:13:37.103751 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [29] name: [add_6] supported: [1] 2023-07-25 15:13:37.103755 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103759 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [30] name: [p_re_lu_7] supported: [0] 2023-07-25 15:13:37.103764 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [31] name: [depthwise_conv2d_7] supported: [1] 2023-07-25 15:13:37.103768 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [32] name: [conv2d_8] supported: [1] 2023-07-25 15:13:37.103774 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [33] name: [add_7] supported: [1] 2023-07-25 15:13:37.103778 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103782 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [34] name: [p_re_lu_8] supported: [0] 2023-07-25 15:13:37.103787 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [35] name: [depthwise_conv2d_8] supported: [1] 2023-07-25 15:13:37.103792 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [36] name: [conv2d_9] supported: [1] 2023-07-25 15:13:37.103796 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [37] name: [add_8] supported: [1] 2023-07-25 15:13:37.103801 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103804 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [38] name: [p_re_lu_9] supported: [0] 2023-07-25 15:13:37.103809 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [MaxPool] index: [39] name: [max_pooling2d_3] supported: [1] 2023-07-25 15:13:37.103814 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Pad] index: [40] name: [channel_padding_3] supported: [0] 2023-07-25 15:13:37.103820 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [41] name: [depthwise_conv2d_9] supported: [1] 2023-07-25 15:13:37.103825 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [42] name: [conv2d_10] supported: [1] 2023-07-25 15:13:37.103830 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [43] name: [add_9] supported: [1] 2023-07-25 15:13:37.103835 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103839 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [44] name: [p_re_lu_10] supported: [0] 2023-07-25 15:13:37.103844 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [45] name: [depthwise_conv2d_10] supported: [1] 2023-07-25 15:13:37.103850 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [46] name: [conv2d_11] supported: [1] 2023-07-25 15:13:37.103857 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [47] name: [add_10] supported: [1] 2023-07-25 15:13:37.103861 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103865 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [48] name: [p_re_lu_11] supported: [0] 2023-07-25 15:13:37.103870 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [49] name: [depthwise_conv2d_11] supported: [1] 2023-07-25 15:13:37.103876 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [50] name: [conv2d_12] supported: [1] 2023-07-25 15:13:37.103880 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [51] name: [add_11] supported: [1] 2023-07-25 15:13:37.103885 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103889 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [52] name: [p_re_lu_12] supported: [0] 2023-07-25 15:13:37.103894 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [MaxPool] index: [53] name: [max_pooling2d_4] supported: [1] 2023-07-25 15:13:37.103899 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [54] name: [depthwise_conv2d_12] supported: [1] 2023-07-25 15:13:37.103904 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [55] name: [conv2d_13] supported: [1] 2023-07-25 15:13:37.103909 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [56] name: [add_12] supported: [1] 2023-07-25 15:13:37.103913 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103918 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [57] name: [p_re_lu_13] supported: [0] 2023-07-25 15:13:37.103923 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [58] name: [depthwise_conv2d_13] supported: [1] 2023-07-25 15:13:37.103927 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [59] name: [conv2d_14] supported: [1] 2023-07-25 15:13:37.103933 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [60] name: [add_13] supported: [1] 2023-07-25 15:13:37.103937 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103941 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [61] name: [p_re_lu_14] supported: [0] 2023-07-25 15:13:37.103946 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [62] name: [depthwise_conv2d_14] supported: [1] 2023-07-25 15:13:37.103951 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [63] name: [conv2d_15] supported: [1] 2023-07-25 15:13:37.103955 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [64] name: [add_14] supported: [1] 2023-07-25 15:13:37.103961 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103965 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [65] name: [p_re_lu_15] supported: [0] 2023-07-25 15:13:37.103970 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [MaxPool] index: [66] name: [max_pooling2d_5] supported: [1] 2023-07-25 15:13:37.103974 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [67] name: [depthwise_conv2d_15] supported: [1] 2023-07-25 15:13:37.103979 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [68] name: [conv2d_16] supported: [1] 2023-07-25 15:13:37.103984 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [69] name: [add_15] supported: [1] 2023-07-25 15:13:37.103989 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.103993 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [70] name: [p_re_lu_16] supported: [0] 2023-07-25 15:13:37.103998 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [71] name: [depthwise_conv2d_16] supported: [1] 2023-07-25 15:13:37.104003 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [72] name: [conv2d_17] supported: [1] 2023-07-25 15:13:37.104008 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [73] name: [add_16] supported: [1] 2023-07-25 15:13:37.104012 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.104017 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [74] name: [p_re_lu_17] supported: [0] 2023-07-25 15:13:37.104022 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [75] name: [depthwise_conv2d_17] supported: [1] 2023-07-25 15:13:37.104027 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [76] name: [conv2d_18] supported: [1] 2023-07-25 15:13:37.104032 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [77] name: [add_17] supported: [1] 2023-07-25 15:13:37.104036 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.104041 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [78] name: [p_re_lu_18] supported: [0] 2023-07-25 15:13:37.104046 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [79] name: [conv2d_19] supported: [1] 2023-07-25 15:13:37.104050 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.104055 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [80] name: [p_re_lu_19] supported: [0] 2023-07-25 15:13:37.104060 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [81] name: [depthwise_conv2d_18] supported: [1] 2023-07-25 15:13:37.104064 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [82] name: [conv2d_20] supported: [1] 2023-07-25 15:13:37.104069 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Add] index: [83] name: [add_18] supported: [1] 2023-07-25 15:13:37.104074 [V:onnxruntime:, activation_op_builder.cc:149 IsPReluOpSupported] PRelu 'slope' input must either have shape [C, 1, 1] or have a single value 2023-07-25 15:13:37.104078 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [PRelu] index: [84] name: [p_re_lu_20] supported: [0] 2023-07-25 15:13:37.104083 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Conv] index: [85] name: [conv2d_21] supported: [1] 2023-07-25 15:13:37.104100 [V:onnxruntime:, helper.cc:97 GetSupportedNodes] Operator type: [Reshape] index: [86] name: [conv2d_21212] supported: [1] 2023-07-25 15:13:37.104186 [W:onnxruntime:, coreml_execution_provider.cc:91 GetCapability] CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 24 number of nodes in the graph: 87 number of nodes supported by CoreML: 64 2023-07-25 15:13:37.105129 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_150] type: [Transpose] was added 2023-07-25 15:13:37.105157 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_1] type: [Conv] was added 2023-07-25 15:13:37.200696 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_1] type: [Conv] was added 2023-07-25 15:13:37.200718 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_2] type: [Conv] was added 2023-07-25 15:13:37.200724 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_1] type: [Add] was added 2023-07-25 15:13:37.235617 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_2] type: [Conv] was added 2023-07-25 15:13:37.235638 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_3] type: [Conv] was added 2023-07-25 15:13:37.235646 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_2] type: [Add] was added 2023-07-25 15:13:37.268791 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_3] type: [Conv] was added 2023-07-25 15:13:37.268812 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_4] type: [Conv] was added 2023-07-25 15:13:37.268824 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [max_pooling2d_1] type: [MaxPool] was added 2023-07-25 15:13:37.298577 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_3] type: [Add] was added 2023-07-25 15:13:37.313610 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_4] type: [Conv] was added 2023-07-25 15:13:37.313627 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_5] type: [Conv] was added 2023-07-25 15:13:37.313633 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_4] type: [Add] was added 2023-07-25 15:13:37.339378 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_5] type: [Conv] was added 2023-07-25 15:13:37.339403 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_6] type: [Conv] was added 2023-07-25 15:13:37.339411 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_5] type: [Add] was added 2023-07-25 15:13:37.363342 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_6] type: [Conv] was added 2023-07-25 15:13:37.363362 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_7] type: [Conv] was added 2023-07-25 15:13:37.363374 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [max_pooling2d_2] type: [MaxPool] was added 2023-07-25 15:13:37.369459 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_6] type: [Add] was added 2023-07-25 15:13:37.384202 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_7] type: [Conv] was added 2023-07-25 15:13:37.384220 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_8] type: [Conv] was added 2023-07-25 15:13:37.384231 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_7] type: [Add] was added 2023-07-25 15:13:37.408780 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_8] type: [Conv] was added 2023-07-25 15:13:37.408799 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_9] type: [Conv] was added 2023-07-25 15:13:37.408805 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_8] type: [Add] was added 2023-07-25 15:13:37.437406 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_9] type: [Conv] was added 2023-07-25 15:13:37.437433 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_10] type: [Conv] was added 2023-07-25 15:13:37.437444 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [max_pooling2d_3] type: [MaxPool] was added 2023-07-25 15:13:37.443400 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_9] type: [Add] was added 2023-07-25 15:13:37.459471 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_10] type: [Conv] was added 2023-07-25 15:13:37.459506 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_11] type: [Conv] was added 2023-07-25 15:13:37.459515 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_10] type: [Add] was added 2023-07-25 15:13:37.489535 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_11] type: [Conv] was added 2023-07-25 15:13:37.489649 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_12] type: [Conv] was added 2023-07-25 15:13:37.489657 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_11] type: [Add] was added 2023-07-25 15:13:37.526022 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_12] type: [Conv] was added 2023-07-25 15:13:37.526059 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_13] type: [Conv] was added 2023-07-25 15:13:37.526072 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [max_pooling2d_4] type: [MaxPool] was added 2023-07-25 15:13:37.526078 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_12] type: [Add] was added 2023-07-25 15:13:37.533552 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_13] type: [Conv] was added 2023-07-25 15:13:37.533582 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_14] type: [Conv] was added 2023-07-25 15:13:37.533588 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_13] type: [Add] was added 2023-07-25 15:13:37.540067 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_14] type: [Conv] was added 2023-07-25 15:13:37.540105 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_15] type: [Conv] was added 2023-07-25 15:13:37.540114 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_14] type: [Add] was added 2023-07-25 15:13:37.546036 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_15] type: [Conv] was added 2023-07-25 15:13:37.546060 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_16] type: [Conv] was added 2023-07-25 15:13:37.546071 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [max_pooling2d_5] type: [MaxPool] was added 2023-07-25 15:13:37.546077 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_15] type: [Add] was added 2023-07-25 15:13:37.552087 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_16] type: [Conv] was added 2023-07-25 15:13:37.552112 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_17] type: [Conv] was added 2023-07-25 15:13:37.552118 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_16] type: [Add] was added 2023-07-25 15:13:37.557808 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_17] type: [Conv] was added 2023-07-25 15:13:37.557827 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_18] type: [Conv] was added 2023-07-25 15:13:37.557833 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_17] type: [Add] was added 2023-07-25 15:13:37.563942 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_19] type: [Conv] was added 2023-07-25 15:13:37.577759 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [depthwise_conv2d_18] type: [Conv] was added 2023-07-25 15:13:37.577776 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_20] type: [Conv] was added 2023-07-25 15:13:37.577782 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [add_18] type: [Add] was added 2023-07-25 15:13:37.582941 [I:onnxruntime:, inference_session.cc:1222 Initialize] Initializing session. 2023-07-25 15:13:37.582961 [I:onnxruntime:, session_state.cc:31 SetupAllocators] Allocator already registered for OrtMemoryInfo:[name:Cpu id:0 OrtMemType:0 OrtAllocatorType:1 Device:[DeviceType:0 MemoryType:0 DeviceId:0]]. Ignoring allocator from CPUExecutionProvider 2023-07-25 15:13:37.583839 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_21] type: [Conv] was added 2023-07-25 15:13:37.583858 [V:onnxruntime:, base_op_builder.cc:52 AddToModelBuilder] Operator name: [conv2d_21212] type: [Reshape] was added 2023-07-25 15:13:37.584443 [I:onnxruntime:, reshape_fusion.cc:42 ApplyImpl] Total fused reshape node count: 0 2023-07-25 15:13:37.584638 [V:onnxruntime:, helper.cc:116 HasNeuralEngine] Current Apple hardware info: arm64 2023-07-25 15:13:37.584646 [W:onnxruntime:, helper.cc:61 IsInputSupported] Dynamic shape is not supported for now, for input:in 2023-07-25 15:13:37.584674 [I:onnxruntime:, coreml_execution_provider.cc:93 GetCapability] CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 0 number of nodes in the graph: 213 number of nodes supported by CoreML: 0 2023-07-25 15:13:37.590487 [V:onnxruntime:, session_state.cc:1010 VerifyEachNodeIsAssignedToAnEp] Node placements 2023-07-25 15:13:37.590500 [V:onnxruntime:, session_state.cc:1013 VerifyEachNodeIsAssignedToAnEp] All nodes placed on [CPUExecutionProvider]. Number of nodes: 213 2023-07-25 15:13:37.590735 [V:onnxruntime:, session_state.cc:66 CreateGraphInfo] SaveMLValueNameIndexMapping 2023-07-25 15:13:37.590788 [V:onnxruntime:, session_state.cc:112 CreateGraphInfo] Done saving OrtValue mappings. 2023-07-25 15:13:37.596353 [I:onnxruntime:, session_state_utils.cc:199 SaveInitializedTensors] Saving initialized tensors. 2023-07-25 15:13:37.598506 [V:onnxruntime:, session_state.cc:1010 VerifyEachNodeIsAssignedToAnEp] Node placements 2023-07-25 15:13:37.598518 [V:onnxruntime:, session_state.cc:1016 VerifyEachNodeIsAssignedToAnEp] Node(s) placed on [CoreMLExecutionProvider]. Number of nodes: 24 2023-07-25 15:13:37.598523 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_0 (CoreMLExecutionProvider_CoreML_18221836647193573136_0_0) 2023-07-25 15:13:37.598528 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_1 (CoreMLExecutionProvider_CoreML_18221836647193573136_1_1) 2023-07-25 15:13:37.598532 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_2 (CoreMLExecutionProvider_CoreML_18221836647193573136_2_2) 2023-07-25 15:13:37.598536 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_3 (CoreMLExecutionProvider_CoreML_18221836647193573136_3_3) 2023-07-25 15:13:37.598540 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_4 (CoreMLExecutionProvider_CoreML_18221836647193573136_4_4) 2023-07-25 15:13:37.598544 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_5 (CoreMLExecutionProvider_CoreML_18221836647193573136_5_5) 2023-07-25 15:13:37.598548 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_6 (CoreMLExecutionProvider_CoreML_18221836647193573136_6_6) 2023-07-25 15:13:37.598552 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_7 (CoreMLExecutionProvider_CoreML_18221836647193573136_7_7) 2023-07-25 15:13:37.598555 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_8 (CoreMLExecutionProvider_CoreML_18221836647193573136_8_8) 2023-07-25 15:13:37.598559 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_9 (CoreMLExecutionProvider_CoreML_18221836647193573136_9_9) 2023-07-25 15:13:37.598563 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_10 (CoreMLExecutionProvider_CoreML_18221836647193573136_10_10) 2023-07-25 15:13:37.598567 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_11 (CoreMLExecutionProvider_CoreML_18221836647193573136_11_11) 2023-07-25 15:13:37.598571 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_12 (CoreMLExecutionProvider_CoreML_18221836647193573136_12_12) 2023-07-25 15:13:37.598574 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_13 (CoreMLExecutionProvider_CoreML_18221836647193573136_13_13) 2023-07-25 15:13:37.598578 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_14 (CoreMLExecutionProvider_CoreML_18221836647193573136_14_14) 2023-07-25 15:13:37.598586 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_15 (CoreMLExecutionProvider_CoreML_18221836647193573136_15_15) 2023-07-25 15:13:37.598590 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_16 (CoreMLExecutionProvider_CoreML_18221836647193573136_16_16) 2023-07-25 15:13:37.598594 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_17 (CoreMLExecutionProvider_CoreML_18221836647193573136_17_17) 2023-07-25 15:13:37.598597 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_18 (CoreMLExecutionProvider_CoreML_18221836647193573136_18_18) 2023-07-25 15:13:37.598601 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_19 (CoreMLExecutionProvider_CoreML_18221836647193573136_19_19) 2023-07-25 15:13:37.598605 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_20 (CoreMLExecutionProvider_CoreML_18221836647193573136_20_20) 2023-07-25 15:13:37.598633 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_21 (CoreMLExecutionProvider_CoreML_18221836647193573136_21_21) 2023-07-25 15:13:37.598638 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_22 (CoreMLExecutionProvider_CoreML_18221836647193573136_22_22) 2023-07-25 15:13:37.598642 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] CoreML_18221836647193573136_23 (CoreMLExecutionProvider_CoreML_18221836647193573136_23_23) 2023-07-25 15:13:37.598646 [V:onnxruntime:, session_state.cc:1016 VerifyEachNodeIsAssignedToAnEp] Node(s) placed on [CPUExecutionProvider]. Number of nodes: 23 2023-07-25 15:13:37.598651 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_1) 2023-07-25 15:13:37.598655 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_2) 2023-07-25 15:13:37.598659 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_3) 2023-07-25 15:13:37.598663 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] Pad (channel_padding_1) 2023-07-25 15:13:37.598667 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_4) 2023-07-25 15:13:37.598671 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_5) 2023-07-25 15:13:37.598675 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_6) 2023-07-25 15:13:37.598679 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] Pad (channel_padding_2) 2023-07-25 15:13:37.598683 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_7) 2023-07-25 15:13:37.598687 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_8) 2023-07-25 15:13:37.598691 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_9) 2023-07-25 15:13:37.598695 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] Pad (channel_padding_3) 2023-07-25 15:13:37.598699 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_10) 2023-07-25 15:13:37.598703 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_11) 2023-07-25 15:13:37.598707 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_12) 2023-07-25 15:13:37.598711 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_13) 2023-07-25 15:13:37.598715 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_14) 2023-07-25 15:13:37.598719 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_15) 2023-07-25 15:13:37.598722 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_16) 2023-07-25 15:13:37.598726 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_17) 2023-07-25 15:13:37.598730 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_18) 2023-07-25 15:13:37.598734 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_19) 2023-07-25 15:13:37.598738 [V:onnxruntime:, session_state.cc:1018 VerifyEachNodeIsAssignedToAnEp] PRelu (p_re_lu_20) 2023-07-25 15:13:37.598829 [V:onnxruntime:, session_state.cc:66 CreateGraphInfo] SaveMLValueNameIndexMapping 2023-07-25 15:13:37.598854 [V:onnxruntime:, session_state.cc:112 CreateGraphInfo] Done saving OrtValue mappings. 2023-07-25 15:13:37.598983 [I:onnxruntime:, session_state_utils.cc:199 SaveInitializedTensors] Saving initialized tensors. 2023-07-25 15:13:37.599461 [I:onnxruntime:, session_state_utils.cc:342 SaveInitializedTensors] Done saving initialized tensors 2023-07-25 15:13:37.599524 [I:onnxruntime:, inference_session.cc:1488 Initialize] Session successfully initialized. 2023-07-25 15:13:37.601751 [I:onnxruntime:, session_state_utils.cc:342 SaveInitializedTensors] Done saving initialized tensors 2023-07-25 15:13:37.601944 [I:onnxruntime:, inference_session.cc:1488 Initialize] Session successfully initialized. 2023-07-25 15:13:38.582414 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:38.686510 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:38.687307 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:38.782663 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:38.783662 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:38.859740 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:38.861441 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:38.923815 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:38.925756 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:38.983028 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:38.983911 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:39.039490 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:39.040741 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:39.094919 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:39.144409 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:44.886884 [I:onnxruntime:, inference_session.cc:1222 Initialize] Initializing session. 2023-07-25 15:13:44.886927 [I:onnxruntime:, session_state.cc:31 SetupAllocators] Allocator already registered for OrtMemoryInfo:[name:Cpu id:0 OrtMemType:0 OrtAllocatorType:1 Device:[DeviceType:0 MemoryType:0 DeviceId:0]]. Ignoring allocator from CPUExecutionProvider 2023-07-25 15:13:44.889241 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_10247'. It is no longer used by any node. 2023-07-25 15:13:44.889255 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_104__49'. It is no longer used by any node. 2023-07-25 15:13:44.889260 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_10853'. It is no longer used by any node. 2023-07-25 15:13:44.889265 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_11658'. It is no longer used by any node. 2023-07-25 15:13:44.889269 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_103__48'. It is no longer used by any node. 2023-07-25 15:13:44.889273 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_11959'. It is no longer used by any node. 2023-07-25 15:13:44.889276 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_10651'. It is no longer used by any node. 2023-07-25 15:13:44.889281 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_125__63'. It is no longer used by any node. 2023-07-25 15:13:44.889285 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_12664'. It is no longer used by any node. 2023-07-25 15:13:44.889289 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_13066'. It is no longer used by any node. 2023-07-25 15:13:44.889293 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_134__68'. It is no longer used by any node. 2023-07-25 15:13:44.889296 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_12965'. It is no longer used by any node. 2023-07-25 15:13:44.889300 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_14274'. It is no longer used by any node. 2023-07-25 15:13:44.889304 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_157__79'. It is no longer used by any node. 2023-07-25 15:13:44.889308 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_19684'. It is no longer used by any node. 2023-07-25 15:13:44.889311 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_13569'. It is no longer used by any node. 2023-07-25 15:13:44.889316 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_101__46'. It is no longer used by any node. 2023-07-25 15:13:44.889320 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_13872'. It is no longer used by any node. 2023-07-25 15:13:44.889324 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_12060'. It is no longer used by any node. 2023-07-25 15:13:44.889329 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_137__71'. It is no longer used by any node. 2023-07-25 15:13:44.889333 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_14575'. It is no longer used by any node. 2023-07-25 15:13:44.889337 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_15177'. It is no longer used by any node. 2023-07-25 15:13:44.889342 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_190__82'. It is no longer used by any node. 2023-07-25 15:13:44.889345 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_12161'. It is no longer used by any node. 2023-07-25 15:13:44.889349 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_13973'. It is no longer used by any node. 2023-07-25 15:13:44.889354 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_124__62'. It is no longer used by any node. 2023-07-25 15:13:44.889358 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_10045'. It is no longer used by any node. 2023-07-25 15:13:44.889361 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_13670'. It is no longer used by any node. 2023-07-25 15:13:44.889366 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_105__50'. It is no longer used by any node. 2023-07-25 15:13:44.889369 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_10752'. It is no longer used by any node. 2023-07-25 15:13:44.889373 [I:onnxruntime:, graph.cc:3490 CleanUnusedInitializersAndNodeArgs] Removing initializer 'Reshape_131__67'. It is no longer used by any node. 2023-07-25 15:13:44.890302 [I:onnxruntime:, reshape_fusion.cc:42 ApplyImpl] Total fused reshape node count: 0 2023-07-25 15:13:44.890678 [I:onnxruntime:, reshape_fusion.cc:42 ApplyImpl] Total fused reshape node count: 0 2023-07-25 15:13:44.890756 [V:onnxruntime:, helper.cc:116 HasNeuralEngine] Current Apple hardware info: arm64 2023-07-25 15:13:44.890767 [W:onnxruntime:, helper.cc:61 IsInputSupported] Dynamic shape is not supported for now, for input:in_face:0 2023-07-25 15:13:44.890793 [I:onnxruntime:, coreml_execution_provider.cc:93 GetCapability] CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 0 number of nodes in the graph: 142 number of nodes supported by CoreML: 0 2023-07-25 15:13:44.891141 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891174 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891186 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891198 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891211 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891221 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891230 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891240 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891250 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891263 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891274 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891284 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891292 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891302 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891311 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891323 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891333 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891342 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891350 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891359 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891369 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891381 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891391 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891400 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891408 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891416 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.891494 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.895237 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.895336 [V:onnxruntime:, selector_action_transformer.cc:129 MatchAndProcess] Matched Conv 2023-07-25 15:13:44.896166 [V:onnxruntime:, session_state.cc:1010 VerifyEachNodeIsAssignedToAnEp] Node placements 2023-07-25 15:13:44.896178 [V:onnxruntime:, session_state.cc:1013 VerifyEachNodeIsAssignedToAnEp] All nodes placed on [CPUExecutionProvider]. Number of nodes: 111 2023-07-25 15:13:44.896354 [V:onnxruntime:, session_state.cc:66 CreateGraphInfo] SaveMLValueNameIndexMapping 2023-07-25 15:13:44.896395 [V:onnxruntime:, session_state.cc:112 CreateGraphInfo] Done saving OrtValue mappings. 2023-07-25 15:13:44.897220 [I:onnxruntime:, session_state_utils.cc:199 SaveInitializedTensors] Saving initialized tensors. 2023-07-25 15:13:45.261807 [I:onnxruntime:, session_state_utils.cc:342 SaveInitializedTensors] Done saving initialized tensors 2023-07-25 15:13:45.404007 [I:onnxruntime:, inference_session.cc:1488 Initialize] Session successfully initialized. 2023-07-25 15:13:45.409184 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:45.412338 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:45.418836 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.34565114974975586 2023-07-25 15:13:45.762798 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:45.764231 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:45.766013 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.33748316764831543 2023-07-25 15:13:46.103497 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:46.107523 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:46.109584 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.32191991806030273 2023-07-25 15:13:46.428478 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:46.431696 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:46.434949 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.326063871383667 2023-07-25 15:13:46.758027 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:46.761174 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:46.762743 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.3824460506439209 2023-07-25 15:13:47.143977 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:47.147762 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:47.152520 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.33515191078186035 2023-07-25 15:13:47.482870 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:47.486949 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:47.488378 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.3151569366455078 2023-07-25 15:13:47.800475 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:47.804915 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:47.806531 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.31312990188598633 2023-07-25 15:13:48.115801 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:48.118541 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:48.120665 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.3171267509460449 2023-07-25 15:13:48.437076 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:48.441256 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:48.443328 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.3099219799041748 2023-07-25 15:13:48.751070 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:48.755203 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:48.758245 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.3237898349761963 2023-07-25 15:13:49.080865 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:49.084352 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:49.086698 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.3256380558013916 2023-07-25 15:13:49.411045 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:49.414144 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:49.416076 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.3165450096130371 2023-07-25 15:13:49.730704 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:49.734061 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:49.735937 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.352567195892334 2023-07-25 15:13:50.094736 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:50.099081 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:50.105752 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.39165639877319336 2023-07-25 15:13:50.490516 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:50.495620 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:50.498348 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.3437769412994385 2023-07-25 15:13:50.837517 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:50.840798 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:50.842634 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.3185160160064697 2023-07-25 15:13:51.159570 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:51.169393 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:51.187829 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.346325159072876 2023-07-25 15:13:51.508474 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:51.512589 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution 2023-07-25 15:13:51.514546 [I:onnxruntime:, sequential_executor.cc:176 Execute] Begin execution Inference time: 0.32715582847595215

.....

I don't know where the problem is, the speed is very slow, each inference needs: 0.352567195892334* 1000 ≈ 305.686 milliseconds

Thanks in advance! 提前感谢！

[cansik]

Could it be because of the supported ops for the CoreML backend? If they are not implemented, the CPU is used, which can lead to a fragmentation of the calculation and data.

[heliumsoft]

When I install this dependency, I got this error. what is a solution?

ERROR: Could not find a version that satisfies the requirement onnxruntime-silicon (from versions: none)
ERROR: No matching distribution found for onnxruntime-silicon

[cansik]

@heliumsoft Instead of writing into a random issue, please read the FAQ first. If there are still questions, open an issue where you describe the problem, the environment you are working in and what you have already tried.

[heliumsoft]

@cansik Thank you. Do you have experience with installing onnxruntime-coreml? Also, can we use onnxruntime-silicon on MacBook Pro Intel Core i7?

[angewandte-codinglab]

Could it be because of the supported ops for the CoreML backend? If they are not implemented, the CPU is used, which can lead to a fragmentation of the calculation and data.

chatGPT thinks so too:

From your log file, it appears that the PRelu and Pad operations are not supported by the CoreML Execution Provider on your device, as indicated by the logs stating they are supported [0]. These operations fall back to the CPU, which can significantly slow down the inference process.

Here are the specific unsupported operations mentioned in the logs:

PRelu: All PRelu operations listed in the logs are marked as unsupported by CoreML. It appears that the slope input for PRelu must either have a shape of [C, 1, 1] or be a single value to be supported, which is not the case in your model. Pad: Specific padding operations (channel_padding_1, channel_padding_2, channel_padding_3) are also listed as unsupported by CoreML. The logs indicate that these Pad operations are not compatible with CoreML's current capabilities.

**To address these unsupported operations:

Modify or Replace Unsupported Operations:** If possible, modify the PRelu layers to meet CoreML's requirements or replace them with other activation functions that are supported by CoreML, such as ReLU. For the Pad operations, consider adjusting the network architecture to reduce or eliminate the need for padding, or adjust the padding parameters to meet the supported configurations if detailed in the CoreML documentation.

Custom Layers: For operations that are essential and cannot be replaced or removed, consider implementing custom layers. However, be aware that this approach requires additional development and may impact performance.

Model Conversion and Simplification: Convert the model to a format more naturally compatible with iOS/macOS development, such as directly using CoreML model format, which might allow more control over how these operations are handled.

Improving compatibility with CoreML can help leverage the full computational capabilities of Apple's hardware, potentially leading to significant performance improvements in your model's inference time.

[cansik]

@angewandte-codinglab Please do not copy/paste what chat-gpt "thinks". Answer questions with your own knowledge and experience.