search

google / uis-rnn

This is the library for the Unbounded Interleaved-State Recurrent Neural Network (UIS-RNN) algorithm, corresponding to the paper Fully Supervised Speaker Diarization.
https://arxiv.org/abs/1810.04719
Apache License 2.0
1.55k stars 320 forks source link

Unable to convert pytorch model to tensorflow in Diarization on mobile device. #79

Closed alamnasim closed 3 years ago

alamnasim commented 3 years ago

Describe the question

I have to run speaker diarization on mobile devices(Android and IOS). I have successfully converted the pytorch based model to ONNX but I am getting some error while converting to tensorflow from ONNX.

"RuntimeError: GRU with linear_before_reset is not supported in Tensorflow." My pytorch version: 1.5.0, ONNX: 1.7.0, onnx-tf: 1.6.1, tensorflow: 2..2.0.

As I can check some of the links below its some sort of onnx and tensorflow compatibility and onnx does not support GRU. https://github.com/onnx/onnx-tensorflow/pull/452 https://github.com/Tencent/ncnn/issues/1745

  1. So I want to know Is there any other way to convert pytorch model to tensorflow or to run it in c++ for inference on mobile device?
  2. Has anyone converted uisrnn model to tensorflow/tflite for mobile device inference?
  3. Do I need to rewrite pytorch based model in tensorflow ?

Please help!

My background

I am looking for speaker diarization on mobile devices for English ascent. I have trained a uisrnn model on TIMIT dataset and now I can do speaker diarization in python using https://github.com/taylorlu/Speaker-Diarization and now I want to convert my uisrnn model to tflite for inference over mobile device.

Have I read the README.md file? yes

Have I searched for similar questions from closed issues?

Have I tried to find the answers in the paper Fully Supervised Speaker Diarization?

Have I tried to find the answers in the reference Speaker Diarization with LSTM?

Have I tried to find the answers in the reference Generalized End-to-End Loss for Speaker Verification?

Thanks

wq2012 commented 3 years ago

Thanks for the question.

This open source library is for prototyping only. It's to help researchers better understand the basic ideas of the paper.

It's NOT for production purposes.

Harry-Garrison commented 3 years ago

@alamnasim Hello! May I ask how you managed to turn the uisrnn model to onnx? I've been trying to do the very same thing and I just can't seem to be able to do it. I have transformed the code in all possible ways and I still can't figure it out. Could you kindly provide some code?