dusty-nv NanoLLM issues

dusty-nv / NanoLLM

Optimized local inference for LLMs with HuggingFace-like APIs for quantization, vision/language models, multimodal agents, speech, vector DB, and RAG.

https://dusty-nv.github.io/NanoLLM/

MIT License

196 stars 31 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Error during quantization step in VideoQuery example on Jetson Orin NX

#53 saarCogni opened 1 day ago
0
Audio Ouput Plugin Question

#52 khalton55 opened 1 week ago
0
Ratelimit Node

#51 JIA-HONG-CHU opened 3 weeks ago
0
RuntimeError: The MD5 Checksum for {path} does not match the expected value.

#50 wow1063490934 opened 3 weeks ago
0
ros_connector implementation

#49 bigrobinson closed 4 weeks ago
2
fixed MLCModel embed_tokens method "input" instead "tokens" var

#48 ms1design opened 1 month ago
0
Getting the same output for the intel realsense camera

#47 chenww05 opened 1 month ago
0
Does AgentStudio allow different functions to be used depending on the input prompt?

#46 shifsa opened 1 month ago
1
avoid SameFileError during restore_config

#45 le-horizon opened 2 months ago
3
Minor tweak to pipertts to allow custom voices

#44 davesarmoury opened 2 months ago
0
Server/Client architecture

#43 WikiLucas00 opened 2 months ago
3
Is it possible to run NanoVLM on data center GPU like A-series, T-series or desktop GPU like 40-series?

#42 frankzflyward opened 2 months ago
1
Where can I get the code of TAM?

#41 PredyDaddy opened 2 months ago
0
system prompt file parsing

#40 bigrobinson closed 2 months ago
1
Steady RAM Usage Increase During Video Inference using video.py

#39 chain-of-immortals opened 2 months ago
7
[Question] Reproducing benchmarks for TinyLlama1.1B

#38 hrishi121 closed 2 months ago
3
Can you add Gemma-2 support by any chance ?

#37 dilerbatu opened 2 months ago
0
Issue: When using image , the response is missing.

#36 vseranvz opened 2 months ago
1
Trying to set system prompt externally with a Python client

#35 lukecdash opened 3 months ago
1
text embedding yields very large multidimensional array

#34 bigrobinson closed 3 months ago
2
Downloads to set external data root

#33 lukecdash closed 3 months ago
0
How can I connect Neo4j graph database with Jetson conainer

#32 jais001 opened 3 months ago
1
[HELP] How to implementa document based RAG with VectorDB using NanoLLM?

#31 jais001 opened 3 months ago
2
[HELP] Can I use local model to load LLM and start the Agent studio ?

#30 lenoardshannon opened 3 months ago
2
fix: pass chat_template as a dict to ChatHistory

#29 ms1design closed 3 months ago
2
all the input arrays must have same number of dimensions, but the array at index 0 has 2 dimension(s) and the array at index 2 has 3 dimension(s)

#28 Shehjad-Ishan opened 3 months ago
2
minor bug in chat history (NameError)

#27 jayant-mil opened 3 months ago
1
feature: Add llama-3.1 chat template with tools support

#26 ms1design closed 3 months ago
3
Question about Video Frame Processing in Live ViLA

#25 YoungjaeDev opened 3 months ago
5
api

#24 tebie6 closed 4 months ago
0
How to support other models?

#23 PredyDaddy closed 4 months ago
3
Import typo in auto_asr.py?

#22 lukecdash opened 4 months ago
2
batch generation is available??

#21 je1lee opened 4 months ago
0
Reorganizing of plugin directories seems to break speech tools such as riva_asr and riva_tts

#20 ChadSwenson opened 4 months ago
1
TTS needs a way to drop inputs from a chat model linked to Video Auto Prompt

#19 TadayukiOkada opened 5 months ago
1
AttributeError: type object 'ChatModel' has no attribute 'OutputEmbed'

#18 TadayukiOkada closed 5 months ago
6
Meta-Llama-3-8B-Instruct always reply: You are a helpful and friendly AI assistant.<|eot_id|>

#17 UserName-wang opened 5 months ago
6
Can I convert a siglip only and not a siglip based LLM?

#16 aliencaocao closed 5 months ago
18
Is have any plan to support low version JetPack, like 4.6.3

#15 yekangming opened 5 months ago
1
ModuleNotFoundError: No module named 'cachetools'

#14 UserName-wang closed 5 months ago
2
support for llama-3-70B

#13 guyo-shifters closed 5 months ago
5
How to know when LLM is done with reply?

#12 ShawnHymel opened 5 months ago
1
Error in riva ASR

#11 ShawnHymel closed 5 months ago
3
Using RTSP as --video-input

#10 zw303 opened 5 months ago
0
Error found when using hf api

#9 Jiopro opened 5 months ago
1
LLaVa 1.6 anyres support

#8 ai-and-i opened 6 months ago
3
Support for LLaVA v1.6?

#7 rsun-bdti opened 6 months ago
6
RuntimeError: phi-2 does not have embed()

#6 ShawnHymel closed 6 months ago
7
NanoVLM live streaming demo works with VILA-2.7b, but not VILA1.5-3b

#5 rsun-bdti closed 6 months ago
6
Stuck at Quantization? or just taking a long time to run?

#4 P15V opened 6 months ago
1