It has been using torchscript models exported by torch.jit.trace() for streaming zipformer.
The issue is that when people want to run the model on GPU, they have to re-export the model using CUDA as the default exported model runs only on CPU.
This PR supports using torch.jit.script() models for streaming zipformer so that users can run the same model both on CPU and GPU.
Hint:
A model exported on CPU by torch.jit.trace() cannot run on GPU
A model exported on CPU by torch.jit.script() is able to run both on CPU and on GPU.
It has been using
torchscript
models exported bytorch.jit.trace()
for streaming zipformer.The issue is that when people want to run the model on GPU, they have to re-export the model using CUDA as the default exported model runs only on CPU.
This PR supports using torch.jit.script() models for streaming zipformer so that users can run the same model both on CPU and GPU.
Hint:
torch.jit.trace()
cannot run on GPUtorch.jit.script()
is able to run both on CPU and on GPU.