Issue while converting model using JIT script

prajwaljpj commented 3 years ago

I was trying to convert my trained model for Hindi to a JIT compatble model using the method provided on the README Facing an error at the dropout layer.

In [6]: phonemizer.predictor.model = torch.jit.script(model)

Unknown type name 'torch.tensor':
  File "/home/gnani/.virtualenvs/ttsapi/lib/python3.8/site-packages/dp/model/utils.py", line 24
    def forward(self, x: torch.tensor) -> torch.tensor:         # shape: [T, N]
                         ~~~~~~~~~~~~ <--- HERE
        x = x + self.scale * self.pe[:x.size(0), :]
        return self.dropout(x)

In eval mode:

ln [8]: model.eval()
In [9]: phonemizer.predictor.model = torch.jit.script(model)

RuntimeError: Can't redefine method: forward on class: __torch__.dp.model.utils.PositionalEncoding

Any suggestions would be greatly appreciated!

prajwaljpj commented 3 years ago

Fixed! Was training the model with the cloned package. Had made some changes in the configs. But was converting model to JIT using the PIP package 'deep-phonemizer'. Although inference of custom model works on PIP package, the torchscript conversion does not work.

Now, I'm facing another issue over the JIT model. During inferencing, the first inference takes 0.64 seconds, and the consecutive inference takes 0.09 to 0.1 seconds.

Any suggestions?

cschaefer26 commented 3 years ago

IMO thats a known issue: https://discuss.pytorch.org/t/speed-of-first-pass-is-very-slow/64575/7

prajwaljpj commented 3 years ago

Interesting! Thank you for your feedback. Closing.

as-ideas / DeepPhonemizer

Issue while converting model using JIT script #9