search

dusty-nv / jetson-containers

Machine Learning Containers for NVIDIA Jetson and JetPack-L4T
MIT License
1.88k stars 416 forks source link

Cannot use model Meta-Llama-3-8B-Instruct lack of config.json file. #550

Open UserName-wang opened 4 weeks ago

UserName-wang commented 4 weeks ago

I applied and downloaded the model files Meta-Llama-3-8B-Instruct via the official script: https://github.com/meta-llama/llama3/blob/main/download.sh. and put the model files in local folder: /data/models/Meta-Llama-3-8B-Instruct/

I tried to load this model but got error message: could not find model config file at .../Meta-Llama-3-8B-Instruct/config.json error

the code to load the model

model = NanoLLM.from_pretrained( model="/data/models/Meta-Llama-3-8B-Instruct", quantization='q4f16_ft', api='mlc' ) but I can use the model: /data/models/Llama-2-7b-chat-hf did I downloaded the all complete files of Meta-Llama-3-8B-Instruct?

dusty-nv commented 4 weeks ago

@UserName-wang you should download the -hf version of it, yea