RuntimeError: Cannot insert a Tensor that requires grad as a constant. Consider making it a parameter or input, or detaching the gradient

[sayakpaul]

Hello.

I am trying to convert the Glow TTS model from PyTorch to ONNX. I am running in to the following issue during the export step -

/content/TTS_repo/TTS/tts/layers/glow_tts/glow.py in forward(self, x, x_mask, reverse, **kwargs)
    261                     self.weight) * (c / self.num_splits) * x_len  # [b]
    262 
--> 263         weight = weight.view(self.num_splits, self.num_splits, 1, 1)
    264         z = F.conv2d(x, weight)
    265 

RuntimeError: Cannot insert a Tensor that requires grad as a constant. Consider making it a parameter or input, or detaching the gradient
Tensor:
-0.4500 -0.2462 -0.0506 -0.4333
-0.3748 -0.0181  0.0442  0.3575
-0.2864  0.7973 -0.0522 -0.1719
 0.1037  0.2156 -0.9611  0.2521
[ torch.FloatTensor{4,4} ]

Here’s my Colab Notebook for the full reproduction of the issue - https://colab.research.google.com/gist/sayakpaul/96f00d33385cbb2c97e1befedcf2e3cd/copy-of-glow_tts.ipynb. Any directions to solve this issue would be greatly helpful.

[erogol]

ONNX does not export custom parameters. weight is manually created.

[erogol]

BTW you might try to export SpeedySpeech model as an alternative. It'd be a nice PR as well.

[sayakpaul]

Cc'ing @tulasiram58827.

Tulasi and I have worked on a couple of TFLite conversions of popular TTS models. We were able to convert GlowTTS from pth to onnx to tflite. But inference is breaking with the converted TFLite model. We have raised an issue on TensorFlow's GitHub as well.

SpeedySpeech seems like a good candidate for our repository that I mentioned above. Will keep you posted if we end up converting it.

[tulasiram58827]

@sayakpaul I tried Speedy Speech also but having some onnx operator issues. Didn't remember exactly what is the issue. Will share once I found the notebook.

[erogol]

Cc'ing @tulasiram58827.

Tulasi and I have worked on a couple of TFLite conversions of popular TTS models. We were able to convert GlowTTS from pth to onnx to tflite. But inference is breaking with the converted TFLite model. We have raised an issue on TensorFlow's GitHub as well.

SpeedySpeech seems like a good candidate for our repository that I mentioned above. Will keep you posted if we end up converting it.

Thx for the update. Have you tried running inference on ONNX for Glow TTS ?

[tulasiram58827]

Yes

[erogol]

and does it work?

[tulasiram58827]

Yes it works

[erogol]

Then I guess you fixed the problem above :)

[tulasiram58827]

Hmm Yes :) . But our end goal is to make inference with TFLite format

[sayakpaul]

Hmm Yes :) . But our end goal is to make inference with TFLite format

Just to expand on this point, it's so because we are broadly interested in exploring the limits of TinyML and EdgeML for that matter. ONNX solves the unification problem pretty well but to make these SoTA models accessible to a wide range of developer audience (like Mobile Application Developers) we believe tools like TensorFlow Lite is necessary.

[neso613]

Cc'ing @tulasiram58827.

Tulasi and I have worked on a couple of TFLite conversions of popular TTS models. We were able to convert GlowTTS from pth to onnx to tflite. But inference is breaking with the converted TFLite model. We have raised an issue on TensorFlow's GitHub as well.

SpeedySpeech seems like a good candidate for our repository that I mentioned above. Will keep you posted if we end up converting it. @sayakpaul You have able to convert glowTTS weeights to onnx. can you pls help me in this regard. can you pls share any reference for this.

[neso613]

Yes it works

@tulasiram58827 I am getting error while onnx inferencing for glowTTS. can you pls help me in this regard

[neso613]

Yes it works

@tulasiram58827 looking forward from hearing from you