search

NVIDIA / nim-anywhere

Accelerate your Gen AI with NVIDIA NIM and NVIDIA AI Workbench
https://www.nvidia.com/en-us/ai/
Apache License 2.0
114 stars 64 forks source link

how to fill-in ENVIRONMENT for Windows version #40

Closed jear closed 1 week ago

jear commented 3 months ago

Hi,

Is this project compatible with AI workbench for Windows ?

If YES, for host mount, what is the Windows "Source Directory" for Docker with WSL setup ? Description What is "Host's runtime files for Docker in Docker support."

Same question for .cache/nvidia-nims

Thx ! Best, Jerome

AmeliaYe commented 3 months ago

Hi Jerome, thank you for the question! Unfortunately, only Ubuntu is supported for remote machines right now.

freemansoft commented 3 months ago

You should post this question here https://forums.developer.nvidia.com/c/ai-data-science/nvidia-ai-workbench/671 There is an AI Workbench forum there and a thread for Nim-Anywhere.

AI Workbench is compatible with Windows in the sense you can run it on Windows.

  1. They install a Linux WSL instance to host AI Workbench.
  2. AI Workbench runs out of that with a user workbench
  3. AI Workbench projects themselves run out of a container with the name of the project. In the case of this project it is something like nim-anywhere
  4. The NIM Anywhere recommended mount in the Environment is to /home/workbench/.cache/nvidia-nims which is on that WSL instance and not your windows drive. This means you can probably share the cache directory across AI Workbench projects because they are all homed off of that WSL instance

You can see the topology here https://joe.blog.freemansoft.com/2024/06/nvidia-ai-studio-workbench-is.html

AmeliaYe commented 3 months ago

Hi! Yes, workbench and this project are compatible with Windows, but NIMs are not. Would that work for you?

jear commented 3 months ago

What I need would be a helm chart, as a matter of fact...

rmkraus commented 3 months ago

We don't have a Helm chart for this particular project because our intention is to make a really accessible place for developers to start learning. However, NVIDIA does have a lot of great work showing more enterprise complete examples.

If you are looking for Helm charts for deploying NVIDIA NIMs, this is a great resource: https://github.com/NVIDIA/nim-deploy

Also our new NIM Agent Blueprints include more enterprise-complete examples including Helm charts: https://blogs.nvidia.com/blog/nim-agent-blueprints/