Speech

[Blakey-Gavin]

1、onnx 转 rknn 时出现大量如下提示： D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7148, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7149, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7151, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7152, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7154, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7155, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7157, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7158, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7160, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7161, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7163, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7164, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7166, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7167, limitation: 2048 D RKNN: [11:46:41.762] DatainEntries overflow, need to tiling more, datain_entries: 7169, limitation: 2048 E RKNN: [11:46:41.762] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.762] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.762] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.762] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.763] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.763] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.763] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.763] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.763] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.763] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.763] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.763] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.763] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 E RKNN: [11:46:41.763] REGTASK: The bit width of field value exceeds the limit, target: lite, offset: 0x1038, shift = 24, limit: 0x1f, value: 0x200 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 18035, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 9017, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4508, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4508, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4509, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4511, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4512, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4514, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4515, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4517, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4518, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4520, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4521, limitation: 2048 D RKNN: [11:46:41.763] DatainEntries overflow, need to tiling more, datain_entries: 4523, limitation: 2048 但是可以生成 rknn 模型

2、在使用生成的 rknn 模型推理时有如下输出： --> exported model successfully! --> Init runtime environment W init_runtime: Target is None, use simulator! done --> Running model GraphPreparing : 100%|██████████████████████████████████████████| 764/764 [00:00<00:00, 3760.62it/s] SessionPreparing : 100%|██████████████████████████████████████████| 764/764 [00:11<00:00, 66.21it/s] W inference: The range [-inf, inf] of '721' is out of the float16! W inference: The range [-inf, inf] of '740' is out of the float16! W inference: The range [-inf, 1.0] of '1317' is out of the float16! W inference: The range [-inf, 1.0] of '1355' is out of the float16! W inference: The range [-inf, inf] of '1721' is out of the float16! W inference: The range [-inf, inf] of '1759' is out of the float16! W inference: The range [-inf, inf] of '1923' is out of the float16! W inference: The range [-inf, inf] of '1961' is out of the float16! done --> Saving result done 模拟推理得到的结果是错误的。我的输入是 (1, 2, 44100) 的语音，1-batch_size，2-立体声，44100-采样点数，如果推理正确输出应该是同样 shape 的人声波形，但是我的推理输出全为 0，所以推理错误。请问从上面看来是什么原因？

[yuyun2000]

从上面看是超出fp16的范围了 不过也不应该输出全0 你看看你推理写的有没有问题吧

[Blakey-Gavin]

从上面看是超出fp16的范围了 不过也不应该输出全0 你看看你推理写的有没有问题吧

推理是直接从 example 里面改的，是没问题的。代码如下： `# Set inputs data, sr = librosa.load(path="1s.wav", sr=None, mono=False) data = np.expand_dims(data, 0)

Init runtime environment

print('--> Init runtime environment') ret = rknn.init_runtime() if ret != 0: print('Init runtime environment failed!') exit(ret) print('done')

Inference

print('--> Running model') outputs = rknn.inference(inputs=[data], data_format=['nchw']) print('done')

Save result

print('--> Saving result') output = np.squeeze(outputs[0]) sf.write("output.wav", output.T, sr, subtype="PCM_16") print('done')

释放 RKNN 对象

rknn.release()`

[yuyun2000]

你输入的三维tensor，应该不需要data_format=['nchw']；你这个问题先确定onnx模型有没有问题，如果onnx没问题可以用analy接口看一下单层精度

[Blakey-Gavin]

好的，我先试试取消掉 data_format 参数。使用 onnxruntime 推理是没有问题的，表明 onnx 没问题对吧，不过我的 onnx 是 fp32 的模型。在转 rknn 时设置的 do_quantization=False，所以是会强制转为 fp16 模型？有没有可能是转换的过程中算子对齐精度的问题呢

另外，我想问一下，前面提到的在转换 rknn 过程中那些提示是什么意思？E 代表的是 Error 吗？

[yuyun2000]

是E是error，一般fp32模型没问题，fp16也是没问题的，极少数比如hubert会出现问题

[Blakey-Gavin]

您好，问题还是存在，您方便帮忙看看吗？

[Blakey-Gavin]

从上面看是超出fp16的范围了 不过也不应该输出全0 你看看你推理写的有没有问题吧

不好意思，我前面说错了。推理的输出不是全 0，是全 nan。如下：

[yuyun2000]

我说了，你用analy接口看看输出精度

[Blakey-Gavin]

谢谢。我使用该接口运行报错如下：

[yuyun2000]

这说明这个模型转出来本身跑的就带nan，先更新rknntoolkit的版本；如果还是一样就要逐层看原网络了，可能是某一层让结果突然变成nan的