dusty-nv NanoLLM issues

dusty-nv / NanoLLM

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.

https://dusty-nv.github.io/NanoLLM/

MIT License

176 stars 26 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

fixed MLCModel embed_tokens method "input" instead "tokens" var

#48 ms1design opened 1 day ago
0
Getting the same output for the intel realsense camera

#47 chenww05 opened 1 week ago
0
Does AgentStudio allow different functions to be used depending on the input prompt?

#46 shifsa opened 1 week ago
1
avoid SameFileError during restore_config

#45 le-horizon opened 2 weeks ago
3
Minor tweak to pipertts to allow custom voices

#44 davesarmoury opened 2 weeks ago
0
Server/Client architecture

#43 WikiLucas00 opened 3 weeks ago
3
Is it possible to run NanoVLM on data center GPU like A-series, T-series or desktop GPU like 40-series?

#42 frankzflyward opened 1 month ago
1
Where can I get the code of TAM?

#41 PredyDaddy opened 1 month ago
0
system prompt file parsing

#40 bigrobinson closed 1 month ago
1
Steady RAM Usage Increase During Video Inference using video.py

#39 chain-of-immortals opened 1 month ago
7
[Question] Reproducing benchmarks for TinyLlama1.1B

#38 hrishi121 closed 1 month ago
3
Can you add Gemma-2 support by any chance ?

#37 dilerbatu opened 1 month ago
0
Issue: When using image , the response is missing.

#36 vseranvz opened 1 month ago
1
Trying to set system prompt externally with a Python client

#35 lukecdash opened 1 month ago
1
text embedding yields very large multidimensional array

#34 bigrobinson closed 1 month ago
2
Downloads to set external data root

#33 lukecdash closed 1 month ago
0
How can I connect Neo4j graph database with Jetson conainer

#32 jais001 opened 1 month ago
1
[HELP] How to implementa document based RAG with VectorDB using NanoLLM?

#31 jais001 opened 1 month ago
2
[HELP] Can I use local model to load LLM and start the Agent studio ?

#30 lenoardshannon opened 1 month ago
2
fix: pass chat_template as a dict to ChatHistory

#29 ms1design closed 1 month ago
2
all the input arrays must have same number of dimensions, but the array at index 0 has 2 dimension(s) and the array at index 2 has 3 dimension(s)

#28 Shehjad-Ishan opened 2 months ago
2
minor bug in chat history (NameError)

#27 jayant-mil opened 2 months ago
1
feature: Add llama-3.1 chat template with tools support

#26 ms1design closed 2 months ago
3
Question about Video Frame Processing in Live ViLA

#25 YoungjaeDev opened 2 months ago
5
api

#24 tebie6 closed 2 months ago
0
How to support other models?

#23 PredyDaddy closed 2 months ago
3
Import typo in auto_asr.py?

#22 lukecdash opened 3 months ago
2
batch generation is available??

#21 je1lee opened 3 months ago
0
Reorganizing of plugin directories seems to break speech tools such as riva_asr and riva_tts

#20 ChadSwenson opened 3 months ago
1
TTS needs a way to drop inputs from a chat model linked to Video Auto Prompt

#19 TadayukiOkada opened 3 months ago
1
AttributeError: type object 'ChatModel' has no attribute 'OutputEmbed'

#18 TadayukiOkada closed 3 months ago
6
Meta-Llama-3-8B-Instruct always reply: You are a helpful and friendly AI assistant.<|eot_id|>

#17 UserName-wang opened 3 months ago
6
Can I convert a siglip only and not a siglip based LLM?

#16 aliencaocao closed 3 months ago
18
Is have any plan to support low version JetPack, like 4.6.3

#15 yekangming opened 4 months ago
1
ModuleNotFoundError: No module named 'cachetools'

#14 UserName-wang closed 4 months ago
2
support for llama-3-70B

#13 guyo-shifters closed 3 months ago
5
How to know when LLM is done with reply?

#12 ShawnHymel opened 4 months ago
1
Error in riva ASR

#11 ShawnHymel closed 4 months ago
3
Using RTSP as --video-input

#10 zw303 opened 4 months ago
0
Error found when using hf api

#9 Jiopro opened 4 months ago
1
LLaVa 1.6 anyres support

#8 ai-and-i opened 4 months ago
3
Support for LLaVA v1.6?

#7 rsun-bdti opened 4 months ago
6
RuntimeError: phi-2 does not have embed()

#6 ShawnHymel closed 4 months ago
7
NanoVLM live streaming demo works with VILA-2.7b, but not VILA1.5-3b

#5 rsun-bdti closed 4 months ago
6
Stuck at Quantization? or just taking a long time to run?

#4 P15V opened 5 months ago
1
Docker Issue / Documentation

#3 bryanhughes closed 4 months ago
3
adding support for nanoLLaVA

#2 qnguyen3 opened 5 months ago
1
pip3 install -r requirements.txt does not work

#1 bryanhughes opened 6 months ago
1