I can load from HF repository, but I can't seem to load from a local path the way I can with hf.
Is this supported?
(I'm using -v with docker to mount a local dir, works fine in -it to, for example, provide access to .py files, so for example I load the below as 'python3 /home/work/models/test.py'
from argparse import ArgumentParser
from huggingface_hub import login
from optimum.nvidia import pipeline
if name == "main":
token="...."
login(token)
model = pipeline("text-generation", model="/home/work/models/Meta-Llama-3-8B-Instruct-fp8", fp8=True)
out = model("What is the latest generation of Nvidia's GPUs?", max_new_tokens=128)
print(out)
I can load from HF repository, but I can't seem to load from a local path the way I can with hf. Is this supported? (I'm using -v with docker to mount a local dir, works fine in -it to, for example, provide access to .py files, so for example I load the below as 'python3 /home/work/models/test.py'
from argparse import ArgumentParser from huggingface_hub import login from optimum.nvidia import pipeline
if name == "main":