search

NVIDIA / NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
https://docs.nvidia.com/nemo-framework/user-guide/latest/overview.html
Apache License 2.0
11.84k stars 2.46k forks source link

How to use/port quartzNet model on mobile? [Question] #1394

Closed rahul-hiker closed 3 years ago

rahul-hiker commented 3 years ago

Can I deploy NeMo trained ASR on mobile platform (Android/iOS)? If yes then what are the steps? Is there a working demo?

okuchaiev commented 3 years ago

Can I deploy NeMo trained ASR on mobile platform (Android/iOS)?

  • Yes!

If yes then what are the steps?

  • Every NeMo model is inherited from torch.nn.Module , therefore you should checkout Pytorch tutorials about deploying models on mobile. For example, you can export QuartzNet ASR models into .onnx file for deployment.

Is there a working demo?

  • Unfortunately, we don't have an open-sourced working demo yet.
rahul-hiker commented 3 years ago

@okuchaiev Thanks for the reply.

I am looking for some answers in order to deploy the NeMo model on mobile.

  1. How to export Nemo quartzNet model to .onnx?
  2. When we deploy it on mobile then what are the libraries required (does it have some dependency on 3rd part libraries apart from pytorch code)?
  3. What will be inference code in mobile?
hoangtuanvu commented 3 years ago

@rahul-hiker you can follow the instruction: https://pytorch.org/mobile/home/ I hope it helps you deploying Nemo model on mobile