search

dusty-nv / NanoLLM

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.
https://dusty-nv.github.io/NanoLLM/
MIT License
196 stars 31 forks source link

Reorganizing of plugin directories seems to break speech tools such as riva_asr and riva_tts #20

Open ChadSwenson opened 4 months ago

ChadSwenson commented 4 months ago

The following commit seems to move several speech related plugins from the audio directory to a new speech directory: Hash: 31ad117619ff12f21f841484fcdf963b1512b159 Message: "updated plugins" Date: 2024/06/12 @ 9:50 PM

Trying to run any example that uses riva asr now give the following error:

Traceback (most recent call last): File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/opt/NanoLLM/nano_llm/agents/web_chat.py", line 327, in agent = WebChat(vars(args)) File "/opt/NanoLLM/nano_llm/agents/web_chat.py", line 32, in init super().init(kwargs) File "/opt/NanoLLM/nano_llm/agents/voice_chat.py", line 38, in init self.asr = AutoASR.from_pretrained(asr=asr, **kwargs) File "/opt/NanoLLM/nano_llm/plugins/speech/auto_asr.py", line 34, in from_pretrained from nano_llm.plugins.audio.riva_asr import RivaASR ModuleNotFoundError: No module named 'nano_llm.plugins.audio.riva_asr'

I am able to resolve this issue for Riva by modifying the imports in the following files: /opt/NanoLLM/nano_llm/plugins/speech/auto_asr.py line 34 /opt/NanoLLM/nano_llm/plugins/speech/auto_tts.py lines 44-46

Changing "audio" to "speech" to match the directory changes in the provided git commit allows the riva asr and tts plugins to work.

dusty-nv commented 4 months ago

Thanks @ChadSwenson , indeed there has been some refactoring of the plugins occurring for Agent Studio - just committed this fix in https://github.com/dusty-nv/NanoLLM/commit/94e690cea759b64da2a9b582f042c8b13aa03172

If you have problems using the container on main branch, you can fallback to running nano_llm container from previous archived release (24.6 was the most recent at this time) - https://hub.docker.com/r/dustynv/nano_llm/tags