Error running with `-m gguf` option

[nickagliano-st]

Hello!

I am trying to run the following command—

llm -m gguf \
  -o path mixtral-8x7b-instruct-v0.1.Q6_K.gguf \
  '[INST] Write a Python function that downloads a file from a URL[/INST]'

—referenced from the tutorial here.

I'm getting this output: Error: 'gguf' is not a known model, which I assume is coming from cli.py.

Some more details:

• Running on an M3 Macbook
• Fresh install of pipx install llm -> llm install llm-llama-cpp -> CMAKE_ARGS="-DLLAMA_METAL=on" FORCE_CMAKE=1 llm install llama-cpp-python
• I can confirm that the mixtral-8x7b-instruct-v0.1.Q6_K.gguf file is present, but it gives the same error message if I just run llm -m gguf, so that's not really related.

Hopefully I'm not missing something obvious. Let me know if I can be of help! I clicked around a bit hoping it might be an easy fix but couldn't find anything.

– NA

[garthk]

27 workaround so -m gguf -o file "$GGUF_MODEL_FILE" will work:

llm llama-cpp models

[nickagliano-st]

Oh, I see. Easy enough. Thanks @garthk!