breizhn / DTLN

Tensorflow 2.x implementation of the DTLN real time speech denoising model. With TF-lite, ONNX and real-time audio processing support.
MIT License
567 stars 160 forks source link

Dimension mismatch error for real_time_dtln_audio.py #60

Closed klausmikealsontheking closed 2 years ago

klausmikealsontheking commented 2 years ago

Hi Nils. I am Kailash. I am trying to implement the real_time_dtln_audio.py for the DTLN_norm_40hr model. so i converted the h5 model into tflite using the code provided in the repo. I modified the code so that the resulting tflite model is dynamically quantized. but when i run the program with the tflite model i got, it is throwing this error

################################################################################ press Return to quit ################################################################################ Exception ignored from cffi callback <function _StreamBase.init..callback_ptr at 0x7f39f4b4dee0>: Traceback (most recent call last): File "/home/bunty/anaconda3/lib/python3.9/site-packages/sounddevice.py", line 880, in callback_ptr return _wrap_callback( File "/home/bunty/anaconda3/lib/python3.9/site-packages/sounddevice.py", line 2681, in _wrap_callback callback(*args) File "/home/bunty/Major/DTLN/real_time_dtln_audio.py", line 107, in callback interpreter_1.set_tensor(input_details_1[0]['index'], in_mag) File "/home/bunty/anaconda3/lib/python3.9/site-packages/tflite_runtime/interpreter.py", line 697, in set_tensor self._interpreter.SetTensor(tensor_index, value) ValueError: Cannot set tensor: Dimension mismatch. Got 3 but expected 4 for input 0.

The real_time_dtln_audio.py is working fine only with the model_1.tflite, model_2.tflite and model_quant_1.tflite and model_quant_2.tflite are used. I tried testing it with a dtln model trained with my personal 10hr dataset(same parameters as given in DNS-Challenge repo), the DTLN_norm_500hr, DTLN_norm_40hr. I got the same error. Even when i created a new tflite file using the model.h5, it gave the same error. Please suggest on how to rectify this issue

klausmikealsontheking commented 2 years ago

https://github.com/breizhn/DTLN/issues/56#issue-1223828128 refer here to resolve this issue