search

NVIDIA / NeMo

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
https://docs.nvidia.com/nemo-framework/user-guide/latest/overview.html
Apache License 2.0
12.06k stars 2.51k forks source link

Request the feature to send metadata in BLS. #4429

Closed ikhyun5538 closed 2 years ago

ikhyun5538 commented 2 years ago

Is your feature request related to a problem? Please describe. I want to change the Model according to the BLS input. But BLS requires the same input bach size. So I have to make input like ["model1", "model1"]. If my batch size is 100, should I send 100 same data like ["model1", "model1", "model1"].

Describe the solution you'd like It would be nice to have the feature to send metadata other than batch data.

titu1994 commented 2 years ago

What is BLS? And what are you trying to accomplish ?

ikhyun5538 commented 2 years ago

Oh i'm sorry. It's Nvidia Triton Issue.

I'm trying to make Korean TTS service using Nemo. It makes a lot of things easier.

thank you.

okuchaiev commented 2 years ago

@ikhyun5538 NVIDIA Riva https://developer.nvidia.com/riva may be of interest to you. You can train conformer-ctc asr models with NeMo then export it to Riva. Riva uses Triton under the hood.

ikhyun5538 commented 2 years ago

@okuchaiev Thanks for the interest and information. It has been very helpful!!