search

NVIDIA / nim-anywhere

Accelerate your Gen AI with NVIDIA NIM and NVIDIA AI Workbench
https://www.nvidia.com/en-us/ai/
Apache License 2.0
111 stars 64 forks source link

LLM NIMs are not properly labeled #37

Open arharpster99 opened 3 months ago

arharpster99 commented 3 months ago

Currently are down as LLM-NIM-0 and LLM-NIM-1, these are actually both the llama3-8B-instruct model and can be viewed within the environment variables, but this should be more accessible.

freemansoft commented 3 months ago

I think they did this so that you could change those LLMs without having to change any of the other code. LLM-NIM-0 and LLM-NIM-1 are just proxies for whatever model you choose.

The default model for LLM-NIM-0 and LLM-NIM-1 should be different models IMO. That would let them show you could use different models for different purposes.