The exported ONNX cannot recognize speech offline. It is normal to use python scripts to infer the pt model

2024-06-29 16:41:14,003 INFO [export-onnx.py:440] {'best_train_loss': inf, 'best_valid_loss': inf, 'best_train_epoch': -1, 'best_valid_epoch': -1, 'batch_idx_train': 0, 'log_interval': 50, 'reset_interval': 200, 'valid_interval': 3000, 'feature_dim': 80, 'subsampling_factor': 4, 'warm_step': 2000, 'env_info': {'k2-version': '1.24.4', 'k2-build-type': 'Release', 'k2-with-cuda': True, 'k2-git-sha1': 'f6919c0ddb311bea7b53a50f3afdcb3c18b8ccc8', 'k2-git-date': 'Sat Feb 10 09:23:09 2024', 'lhotse-version': '1.24.1', 'torch-version': '1.11.0+cu113', 'torch-cuda-available': True, 'torch-cuda-version': '11.3', 'python-version': '3.8', 'icefall-git-branch': None, 'icefall-git-sha1': None, 'icefall-git-date': None, 'icefall-path': '/home/kell/anaconda3/envs/icefall/lib/python3.8/site-packages/icefall-1.0-py3.8.egg', 'k2-path': '/home/kell/anaconda3/envs/icefall/lib/python3.8/site-packages/k2/init.py', 'lhotse-path': '/home/kell/anaconda3/envs/icefall/lib/python3.8/site-packages/lhotse/init.py', 'hostname': '192-168-98-170', 'IP address': '192.168.98.170'}, 'epoch': 3, 'iter': 0, 'avg': 2, 'use_averaged_model': False, 'exp_dir': PosixPath('exp/zipformer_onnx'), 'tokens': './data/lang_char/tokens.txt', 'context_size': 2, 'num_encoder_layers': '2,2,3,4,3,2', 'downsampling_factor': '1,2,4,8,4,2', 'feedforward_dim': '512,768,1024,1536,1024,768', 'num_heads': '4,4,4,8,4,4', 'encoder_dim': '192,256,384,512,384,256', 'query_head_dim': '32', 'value_head_dim': '12', 'pos_head_dim': '4', 'pos_dim': 48, 'encoder_unmasked_dim': '192,192,256,256,256,192', 'cnn_module_kernel': '31,31,15,15,15,31', 'decoder_dim': 512, 'joiner_dim': 512, 'causal': False, 'chunk_size': '16,32,64,-1', 'left_context_frames': '64,128,256,-1', 'use_transducer': True, 'use_ctc': False, 'blank_id': 0, 'vocab_size': 4362} 2024-06-29 16:41:14,004 INFO [export-onnx.py:442] About to create model 2024-06-29 16:41:20,578 INFO [export-onnx.py:473] averaging ['exp/zipformer_onnx/epoch-2.pt', 'exp/zipformer_onnx/epoch-3.pt'] 2024-06-29 16:41:23,700 INFO [export-onnx.py:546] encoder parameters: 64256759 2024-06-29 16:41:23,700 INFO [export-onnx.py:547] decoder parameters: 2500096 2024-06-29 16:41:23,700 INFO [export-onnx.py:548] joiner parameters: 2237706 2024-06-29 16:41:23,700 INFO [export-onnx.py:549] total parameters: 68994561 2024-06-29 16:41:23,700 INFO [export-onnx.py:560] Exporting encoder /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/scaling.py:1486: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. zero = torch.tensor(0.0, dtype=x.dtype, device=x.device) /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/scaling.py:1412: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect. zero = torch.tensor(0.0, dtype=x.dtype, device=x.device) /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/scaling.py:473: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert x.shape[self.channel_dim] == self.num_channels /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/subsampling.py:325: UserWarning: floordiv is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). x_lens = (x_lens - 7) // 2 /home/kell/anaconda3/envs/icefall/lib/python3.8/site-packages/icefall-1.0-py3.8.egg/icefall/utils.py:1357: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! max_len = max(max_len, lengths.max()) /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/scaling.py:1657: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! if num_channels <= x.shape[-1]: /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/zipformer.py:1612: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert p.shape[-1] == num_heads pos_head_dim, (p.shape[-1], num_heads, pos_head_dim) /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/zipformer.py:1692: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert attn_scores.shape == (num_heads, batch_size, seq_len, seq_len) /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/zipformer.py:1703: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert key_padding_mask.shape == ( /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/zipformer.py:2120: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert attn_weights.shape == (num_heads, batch_size, seq_len, seq_len) /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/zipformer.py:1910: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert attn_weights.shape == (num_heads, batch_size, seq_len, seq_len) /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/zipformer.py:1331: UserWarning: floordiv is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). d_seq_len = (seq_len + ds - 1) // ds /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/zipformer.py:1338: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs! assert src.shape[0] == d_seq_len ds /170-asr/kell/backstreet/icefall-master/egs/librispeech/ASR/zipformer/zipformer.py:356: UserWarning: floordiv is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor'). lengths = (x_lens + 1) // 2 /home/kell/anaconda3/envs/icefall/lib/python3.8/site-packages/torch/onnx/utils.py:359: UserWarning: Model has no forward function warnings.warn("Model has no forward function") 2024-06-29 16:42:51,006 INFO [export-onnx.py:324] meta_data: {'model_type': 'zipformer2', 'version': '1', 'model_author': 'k2-fsa', 'comment': 'non-streaming zipformer2'} 2024-06-29 16:42:52,090 INFO [export-onnx.py:567] Exported encoder to exp/zipformer_onnx/encoder-epoch-3-avg-2.onnx 2024-06-29 16:42:52,091 INFO [export-onnx.py:569] Exporting decoder WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. WARNING: The shape inference of prim::Constant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. 2024-06-29 16:42:52,275 INFO [export-onnx.py:576] Exported decoder to exp/zipformer_onnx/decoder-epoch-3-avg-2.onnx 2024-06-29 16:42:52,275 INFO [export-onnx.py:578] Exporting joiner 2024-06-29 16:42:52,275 INFO [export-onnx.py:394] joiner dim: 512 2024-06-29 16:42:52,323 INFO [export-onnx.py:585] Exported joiner to exp/zipformer_onnx/joiner-epoch-3-avg-2.onnx 2024-06-29 16:42:52,323 INFO [export-onnx.py:590] Generate int8 quantization models 2024-06-29 16:42:53,905 WARNING [quantize.py:637] Please consider to run pre-processing before quantization. Refer to example: https://github.com/microsoft/onnxruntime-inference-examples/blob/main/quantization/image_classification/cpu/ReadMe.md 2024-06-29 16:42:54,568 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.23" not specified 2024-06-29 16:42:54,578 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.27" not specified 2024-06-29 16:42:54,581 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_434] 2024-06-29 16:42:54,585 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"pos_emb" not specified 2024-06-29 16:42:54,587 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_445] 2024-06-29 16:42:54,594 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.52" not specified 2024-06-29 16:42:54,600 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.35" not specified 2024-06-29 16:42:54,603 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_580] 2024-06-29 16:42:54,606 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.39" not specified 2024-06-29 16:42:54,612 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.76" not specified 2024-06-29 16:42:54,614 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_615] 2024-06-29 16:42:54,618 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.43" not specified 2024-06-29 16:42:54,623 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.47" not specified 2024-06-29 16:42:54,629 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.112" not specified 2024-06-29 16:42:54,634 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.55" not specified 2024-06-29 16:42:54,640 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.120" not specified 2024-06-29 16:42:54,646 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.124" not specified 2024-06-29 16:42:54,648 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_716] 2024-06-29 16:42:54,652 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.59" not specified 2024-06-29 16:42:54,657 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.63" not specified 2024-06-29 16:42:54,790 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.160" not specified 2024-06-29 16:42:54,797 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.71" not specified 2024-06-29 16:42:54,804 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.168" not specified 2024-06-29 16:42:54,810 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.75" not specified 2024-06-29 16:42:54,813 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_850] 2024-06-29 16:42:54,816 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_861] 2024-06-29 16:42:54,823 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.188" not specified 2024-06-29 16:42:54,829 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.83" not specified 2024-06-29 16:42:54,832 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_994] 2024-06-29 16:42:54,836 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.87" not specified 2024-06-29 16:42:54,841 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.212" not specified 2024-06-29 16:42:54,844 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_1029] 2024-06-29 16:42:54,847 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.91" not specified 2024-06-29 16:42:54,853 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.95" not specified 2024-06-29 16:42:54,859 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.248" not specified 2024-06-29 16:42:54,864 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.103" not specified 2024-06-29 16:42:54,870 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.256" not specified 2024-06-29 16:42:54,876 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.260" not specified 2024-06-29 16:42:54,879 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_1130] 2024-06-29 16:42:54,883 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.107" not specified 2024-06-29 16:42:54,888 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.111" not specified 2024-06-29 16:42:54,894 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.296" not specified 2024-06-29 16:42:54,900 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.119" not specified 2024-06-29 16:42:54,906 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.304" not specified 2024-06-29 16:42:54,913 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.123" not specified 2024-06-29 16:42:54,916 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_1580] 2024-06-29 16:42:54,920 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"pos_emb.19" not specified 2024-06-29 16:42:54,922 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_1591] 2024-06-29 16:42:54,930 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.332" not specified 2024-06-29 16:42:54,936 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.131" not specified 2024-06-29 16:42:54,940 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_1726] 2024-06-29 16:42:54,943 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.135" not specified 2024-06-29 16:42:54,949 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.356" not specified 2024-06-29 16:42:54,952 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_1761] 2024-06-29 16:42:54,956 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.139" not specified 2024-06-29 16:42:54,962 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.143" not specified 2024-06-29 16:42:54,968 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.392" not specified 2024-06-29 16:42:54,974 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.151" not specified 2024-06-29 16:42:54,981 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.400" not specified 2024-06-29 16:42:54,988 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.404" not specified 2024-06-29 16:42:54,991 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_1862] 2024-06-29 16:42:54,994 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.155" not specified 2024-06-29 16:42:55,110 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.159" not specified 2024-06-29 16:42:55,118 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.440" not specified 2024-06-29 16:42:55,125 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.167" not specified 2024-06-29 16:42:55,133 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.448" not specified 2024-06-29 16:42:55,140 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.171" not specified 2024-06-29 16:42:55,144 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_1996] 2024-06-29 16:42:55,148 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_2007] 2024-06-29 16:42:55,157 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.468" not specified 2024-06-29 16:42:55,165 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.179" not specified 2024-06-29 16:42:55,169 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_2140] 2024-06-29 16:42:55,173 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.183" not specified 2024-06-29 16:42:55,180 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.492" not specified 2024-06-29 16:42:55,183 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_2175] 2024-06-29 16:42:55,187 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.187" not specified 2024-06-29 16:42:55,193 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.191" not specified 2024-06-29 16:42:55,201 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.528" not specified 2024-06-29 16:42:55,208 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.199" not specified 2024-06-29 16:42:55,216 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.536" not specified 2024-06-29 16:42:55,224 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.540" not specified 2024-06-29 16:42:55,227 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_2276] 2024-06-29 16:42:55,231 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.203" not specified 2024-06-29 16:42:55,238 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.207" not specified 2024-06-29 16:42:55,245 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.576" not specified 2024-06-29 16:42:55,252 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.215" not specified 2024-06-29 16:42:55,260 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.584" not specified 2024-06-29 16:42:55,269 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.219" not specified 2024-06-29 16:42:55,274 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_2770] 2024-06-29 16:42:55,278 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"pos_emb.39" not specified 2024-06-29 16:42:55,281 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_2781] 2024-06-29 16:42:55,291 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.612" not specified 2024-06-29 16:42:55,300 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.227" not specified 2024-06-29 16:42:55,305 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_2916] 2024-06-29 16:42:55,309 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.231" not specified 2024-06-29 16:42:55,316 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.636" not specified 2024-06-29 16:42:55,319 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_2951] 2024-06-29 16:42:55,323 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.235" not specified 2024-06-29 16:42:55,330 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.239" not specified 2024-06-29 16:42:55,338 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.672" not specified 2024-06-29 16:42:55,346 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.247" not specified 2024-06-29 16:42:55,355 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.680" not specified 2024-06-29 16:42:55,364 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.684" not specified 2024-06-29 16:42:55,367 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_3052] 2024-06-29 16:42:55,371 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.251" not specified 2024-06-29 16:42:55,378 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.255" not specified 2024-06-29 16:42:55,386 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.720" not specified 2024-06-29 16:42:55,394 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.263" not specified 2024-06-29 16:42:55,404 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.728" not specified 2024-06-29 16:42:55,415 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.267" not specified 2024-06-29 16:42:55,419 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_3186] 2024-06-29 16:42:55,424 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_3197] 2024-06-29 16:42:55,434 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.748" not specified 2024-06-29 16:42:55,442 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.275" not specified 2024-06-29 16:42:55,447 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_3330] 2024-06-29 16:42:55,451 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.279" not specified 2024-06-29 16:42:55,459 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.772" not specified 2024-06-29 16:42:55,462 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_3365] 2024-06-29 16:42:55,466 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.283" not specified 2024-06-29 16:42:55,473 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.287" not specified 2024-06-29 16:42:55,481 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.808" not specified 2024-06-29 16:42:55,489 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.295" not specified 2024-06-29 16:42:55,498 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.816" not specified 2024-06-29 16:42:55,507 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.820" not specified 2024-06-29 16:42:55,511 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_3466] 2024-06-29 16:42:55,515 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.299" not specified 2024-06-29 16:42:55,521 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.303" not specified 2024-06-29 16:42:55,530 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.856" not specified 2024-06-29 16:42:55,537 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.311" not specified 2024-06-29 16:42:55,548 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.864" not specified 2024-06-29 16:42:55,558 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.315" not specified 2024-06-29 16:42:55,562 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_3600] 2024-06-29 16:42:55,566 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_3611] 2024-06-29 16:42:55,577 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.884" not specified 2024-06-29 16:42:55,586 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.323" not specified 2024-06-29 16:42:55,591 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_3744] 2024-06-29 16:42:55,702 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.327" not specified 2024-06-29 16:42:55,711 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.908" not specified 2024-06-29 16:42:55,715 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_3779] 2024-06-29 16:42:55,719 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.331" not specified 2024-06-29 16:42:55,726 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.335" not specified 2024-06-29 16:42:55,735 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.944" not specified 2024-06-29 16:42:55,743 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.343" not specified 2024-06-29 16:42:55,752 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.952" not specified 2024-06-29 16:42:55,761 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.956" not specified 2024-06-29 16:42:55,765 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_3880] 2024-06-29 16:42:55,769 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.347" not specified 2024-06-29 16:42:55,776 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.351" not specified 2024-06-29 16:42:55,785 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.992" not specified 2024-06-29 16:42:55,792 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.359" not specified 2024-06-29 16:42:55,803 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1000" not specified 2024-06-29 16:42:55,813 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.363" not specified 2024-06-29 16:42:55,819 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_4374] 2024-06-29 16:42:55,823 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"pos_emb.67" not specified 2024-06-29 16:42:55,827 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_4385] 2024-06-29 16:42:55,839 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1028" not specified 2024-06-29 16:42:55,850 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.371" not specified 2024-06-29 16:42:55,857 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_4520] 2024-06-29 16:42:55,862 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.375" not specified 2024-06-29 16:42:55,870 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1052" not specified 2024-06-29 16:42:55,874 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_4555] 2024-06-29 16:42:55,878 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.379" not specified 2024-06-29 16:42:55,885 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.383" not specified 2024-06-29 16:42:55,897 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1088" not specified 2024-06-29 16:42:55,906 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.391" not specified 2024-06-29 16:42:55,921 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1096" not specified 2024-06-29 16:42:55,934 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1100" not specified 2024-06-29 16:42:55,938 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_4656] 2024-06-29 16:42:55,942 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.395" not specified 2024-06-29 16:42:55,949 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.399" not specified 2024-06-29 16:42:55,960 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1136" not specified 2024-06-29 16:42:55,969 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.407" not specified 2024-06-29 16:42:55,985 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1144" not specified 2024-06-29 16:42:55,998 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.411" not specified 2024-06-29 16:42:56,003 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_4790] 2024-06-29 16:42:56,008 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_4801] 2024-06-29 16:42:56,022 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1164" not specified 2024-06-29 16:42:56,033 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.419" not specified 2024-06-29 16:42:56,039 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_4934] 2024-06-29 16:42:56,044 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.423" not specified 2024-06-29 16:42:56,052 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1188" not specified 2024-06-29 16:42:56,057 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_4969] 2024-06-29 16:42:56,061 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.427" not specified 2024-06-29 16:42:56,068 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.431" not specified 2024-06-29 16:42:56,078 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1224" not specified 2024-06-29 16:42:56,087 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.439" not specified 2024-06-29 16:42:56,100 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1232" not specified 2024-06-29 16:42:56,111 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1236" not specified 2024-06-29 16:42:56,116 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_5070] 2024-06-29 16:42:56,120 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.443" not specified 2024-06-29 16:42:56,127 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.447" not specified 2024-06-29 16:42:56,138 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1272" not specified 2024-06-29 16:42:56,147 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.455" not specified 2024-06-29 16:42:56,162 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1280" not specified 2024-06-29 16:42:56,176 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.459" not specified 2024-06-29 16:42:56,182 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_5204] 2024-06-29 16:42:56,187 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_5215] 2024-06-29 16:42:56,201 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1300" not specified 2024-06-29 16:42:56,214 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.467" not specified 2024-06-29 16:42:56,222 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_5348] 2024-06-29 16:42:56,227 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.471" not specified 2024-06-29 16:42:56,236 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1324" not specified 2024-06-29 16:42:56,241 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_5383] 2024-06-29 16:42:56,245 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.475" not specified 2024-06-29 16:42:56,253 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.479" not specified 2024-06-29 16:42:56,268 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1360" not specified 2024-06-29 16:42:56,278 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.487" not specified 2024-06-29 16:42:56,292 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1368" not specified 2024-06-29 16:42:56,306 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1372" not specified 2024-06-29 16:42:56,311 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_5484] 2024-06-29 16:42:56,315 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.491" not specified 2024-06-29 16:42:56,324 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.495" not specified 2024-06-29 16:42:56,337 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1408" not specified 2024-06-29 16:42:56,348 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.503" not specified 2024-06-29 16:42:56,365 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1416" not specified 2024-06-29 16:42:56,382 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.507" not specified 2024-06-29 16:42:56,389 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_5618] 2024-06-29 16:42:56,395 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_5629] 2024-06-29 16:42:56,410 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1436" not specified 2024-06-29 16:42:56,423 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.515" not specified 2024-06-29 16:42:56,433 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_5762] 2024-06-29 16:42:56,438 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.519" not specified 2024-06-29 16:42:56,448 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1460" not specified 2024-06-29 16:42:56,453 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_5797] 2024-06-29 16:42:56,457 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.523" not specified 2024-06-29 16:42:56,466 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.527" not specified 2024-06-29 16:42:56,479 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1496" not specified 2024-06-29 16:42:56,489 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.535" not specified 2024-06-29 16:42:56,504 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1504" not specified 2024-06-29 16:42:56,519 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1508" not specified 2024-06-29 16:42:56,524 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_5898] 2024-06-29 16:42:56,528 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.539" not specified 2024-06-29 16:42:56,536 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.543" not specified 2024-06-29 16:42:56,548 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1544" not specified 2024-06-29 16:42:56,559 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.551" not specified 2024-06-29 16:42:56,576 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1552" not specified 2024-06-29 16:42:56,593 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.555" not specified 2024-06-29 16:42:56,598 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_6372] 2024-06-29 16:42:56,603 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"pos_emb.103" not specified 2024-06-29 16:42:56,607 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_6383] 2024-06-29 16:42:56,618 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1580" not specified 2024-06-29 16:42:56,628 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.563" not specified 2024-06-29 16:42:56,634 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_6516] 2024-06-29 16:42:56,639 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.567" not specified 2024-06-29 16:42:56,648 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1604" not specified 2024-06-29 16:42:56,652 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_6551] 2024-06-29 16:42:56,657 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.571" not specified 2024-06-29 16:42:56,665 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.575" not specified 2024-06-29 16:42:56,676 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1640" not specified 2024-06-29 16:42:56,685 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.583" not specified 2024-06-29 16:42:56,695 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1648" not specified 2024-06-29 16:42:56,706 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1652" not specified 2024-06-29 16:42:56,710 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_6652] 2024-06-29 16:42:56,715 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.587" not specified 2024-06-29 16:42:56,723 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.591" not specified 2024-06-29 16:42:56,733 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1688" not specified 2024-06-29 16:42:56,742 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.599" not specified 2024-06-29 16:42:56,755 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1696" not specified 2024-06-29 16:42:56,767 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.603" not specified 2024-06-29 16:42:56,773 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_6786] 2024-06-29 16:42:56,779 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_6797] 2024-06-29 16:42:56,791 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1716" not specified 2024-06-29 16:42:56,801 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.611" not specified 2024-06-29 16:42:56,807 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_6930] 2024-06-29 16:42:56,812 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.615" not specified 2024-06-29 16:42:56,821 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1740" not specified 2024-06-29 16:42:56,826 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_6965] 2024-06-29 16:42:56,830 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.619" not specified 2024-06-29 16:42:56,839 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.623" not specified 2024-06-29 16:42:56,849 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1776" not specified 2024-06-29 16:42:56,857 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.631" not specified 2024-06-29 16:42:56,868 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1784" not specified 2024-06-29 16:42:56,879 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1788" not specified 2024-06-29 16:42:56,884 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_7066] 2024-06-29 16:42:56,888 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.635" not specified 2024-06-29 16:42:56,896 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.639" not specified 2024-06-29 16:42:56,907 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1824" not specified 2024-06-29 16:42:56,916 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.647" not specified 2024-06-29 16:42:56,928 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1832" not specified 2024-06-29 16:42:56,939 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.651" not specified 2024-06-29 16:42:56,945 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_7200] 2024-06-29 16:42:56,951 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_7211] 2024-06-29 16:42:56,963 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1852" not specified 2024-06-29 16:42:56,973 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.659" not specified 2024-06-29 16:42:56,979 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_7344] 2024-06-29 16:42:56,984 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.663" not specified 2024-06-29 16:42:56,993 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1876" not specified 2024-06-29 16:42:56,998 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_7379] 2024-06-29 16:42:57,002 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.667" not specified 2024-06-29 16:42:57,011 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.671" not specified 2024-06-29 16:42:57,021 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1912" not specified 2024-06-29 16:42:57,030 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.679" not specified 2024-06-29 16:42:57,040 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1920" not specified 2024-06-29 16:42:57,051 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1924" not specified 2024-06-29 16:42:57,056 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_7480] 2024-06-29 16:42:57,061 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.683" not specified 2024-06-29 16:42:57,070 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.687" not specified 2024-06-29 16:42:57,082 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1960" not specified 2024-06-29 16:42:57,091 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.695" not specified 2024-06-29 16:42:57,104 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1968" not specified 2024-06-29 16:42:57,116 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.699" not specified 2024-06-29 16:42:57,121 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_7954] 2024-06-29 16:42:57,126 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"pos_emb.131" not specified 2024-06-29 16:42:57,131 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_7965] 2024-06-29 16:42:57,142 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.1996" not specified 2024-06-29 16:42:57,151 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.707" not specified 2024-06-29 16:42:57,157 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_8098] 2024-06-29 16:42:57,162 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.711" not specified 2024-06-29 16:42:57,170 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2020" not specified 2024-06-29 16:42:57,175 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_8133] 2024-06-29 16:42:57,180 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.715" not specified 2024-06-29 16:42:57,188 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.719" not specified 2024-06-29 16:42:57,198 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2056" not specified 2024-06-29 16:42:57,206 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.727" not specified 2024-06-29 16:42:57,216 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2064" not specified 2024-06-29 16:42:57,225 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2068" not specified 2024-06-29 16:42:57,231 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_8234] 2024-06-29 16:42:57,235 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.731" not specified 2024-06-29 16:42:57,244 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.735" not specified 2024-06-29 16:42:57,253 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2104" not specified 2024-06-29 16:42:57,262 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.743" not specified 2024-06-29 16:42:57,272 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2112" not specified 2024-06-29 16:42:57,282 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.747" not specified 2024-06-29 16:42:57,287 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_8368] 2024-06-29 16:42:57,294 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_8379] 2024-06-29 16:42:57,305 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2132" not specified 2024-06-29 16:42:57,314 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.755" not specified 2024-06-29 16:42:57,320 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_8512] 2024-06-29 16:42:57,325 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.759" not specified 2024-06-29 16:42:57,333 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2156" not specified 2024-06-29 16:42:57,339 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_8547] 2024-06-29 16:42:57,343 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.763" not specified 2024-06-29 16:42:57,352 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.767" not specified 2024-06-29 16:42:57,362 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2192" not specified 2024-06-29 16:42:57,371 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.775" not specified 2024-06-29 16:42:57,382 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2200" not specified 2024-06-29 16:42:57,392 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2204" not specified 2024-06-29 16:42:57,398 INFO [matmul.py:30] Ignore MatMul due to non constant B: /[MatMul_8648] 2024-06-29 16:42:57,402 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.779" not specified 2024-06-29 16:42:57,411 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.783" not specified 2024-06-29 16:42:57,421 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2240" not specified 2024-06-29 16:42:57,431 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.791" not specified 2024-06-29 16:42:57,442 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"x.2248" not specified 2024-06-29 16:42:57,453 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"input.795" not specified 2024-06-29 16:42:58,084 WARNING [quantize.py:637] Please consider to run pre-processing before quantization. Refer to example: https://github.com/microsoft/onnxruntime-inference-examples/blob/main/quantization/image_classification/cpu/ReadMe.md 2024-06-29 16:42:58,115 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"decoder_output" not specified 2024-06-29 16:42:58,160 WARNING [quantize.py:637] Please consider to run pre-processing before quantization. Refer to example: https://github.com/microsoft/onnxruntime-inference-examples/blob/main/quantization/image_classification/cpu/ReadMe.md 2024-06-29 16:42:58,211 INFO [onnx_quantizer.py:513] Quantization parameters for tensor:"onnx::Gemm_5" not specified

/170-asr/kell/backstreet/sherpa-onnx-master/sherpa-onnx/csrc/parse-options.cc:Read:360 ./sherpa-onnx-offline --tokens=./onnx_t_epoch3/tokens.txt --encoder=./onnx_t_epoch3/encoder-epoch-3-avg-2.onnx --decoder=./onnx_t_epoch3/decoder-epoch-3-avg-2.onnx --joiner=./onnx_t_epoch3/joiner-epoch-3-avg-2.onnx --provider=cpu A2_0.wav

OfflineRecognizerConfig(feat_config=FeatureExtractorConfig(sampling_rate=16000, feature_dim=80, low_freq=20, high_freq=-400, dither=0), model_config=OfflineModelConfig(transducer=OfflineTransducerModelConfig(encoder_filename="./onnx_t_epoch3/encoder-epoch-3-avg-2.onnx", decoder_filename="./onnx_t_epoch3/decoder-epoch-3-avg-2.onnx", joiner_filename="./onnx_t_epoch3/joiner-epoch-3-avg-2.onnx"), paraformer=OfflineParaformerModelConfig(model=""), nemo_ctc=OfflineNemoEncDecCtcModelConfig(model=""), whisper=OfflineWhisperModelConfig(encoder="", decoder="", language="", task="transcribe", tail_paddings=-1), tdnn=OfflineTdnnModelConfig(model=""), zipformer_ctc=OfflineZipformerCtcModelConfig(model=""), wenet_ctc=OfflineWenetCtcModelConfig(model=""), telespeech_ctc="", tokens="./onnx_t_epoch3/tokens.txt", num_threads=2, debug=False, provider="cpu", model_type="", modeling_unit="cjkchar", bpe_vocab=""), lm_config=OfflineLMConfig(model="", scale=0.5), ctc_fst_decoder_config=OfflineCtcFstDecoderConfig(graph="", max_active=3000), decoding_method="greedy_search", max_active_paths=4, hotwords_file="", hotwords_score=1.5, blank_penalty=0, rule_fsts="", rule_fars="") Creating recognizer ... Started Done!

A2_0.wav {"text": "嗯什么", "timestamps": [0.04, 8.16, 8.36], "tokens":["嗯", "什", "么"], "words": []}

num threads: 2 decoding method: greedy_search Elapsed seconds: 0.407 s Real time factor (RTF): 0.407 / 9.812 = 0.041

k2-fsa / icefall

The exported ONNX cannot recognize speech offline. It is normal to use python scripts to infer the pt model #1673

A2_0.wav {"text": "嗯什么", "timestamps": [0.04, 8.16, 8.36], "tokens":["嗯", "什", "么"], "words": []}