Allow Ollama to run other models besides the llama family

[gfxenjoyer]

I managed to run other models like gemma2 and phi3.5 by changing the lines

# TODO: Eventually this will move to the llama cli model list command
# mapping of Model SKUs to ollama models
OLLAMA_SUPPORTED_SKUS = {
    # "Llama3.1-8B-Instruct": "llama3.1",
    "Llama3.1-8B-Instruct": "gemma2:latest",
    "Llama3.1-70B-Instruct": "llama3.1:70b-instruct-fp16",
}

in ./llama_stack/providers/adapters/inference/ollama.py.

Can the Ollama provider give us an escape hatch to easily run Ollama models?

Thanks

[bhargavyagnik]

If you dont mind, i tried doing the ollama inference, but cant get to execute the stack build. can you suggest me what steps you followed ?

[kavukcutolga]

I'm also interested to hear how we can run any ollama models?

[raghotham]

cant get to execute the stack build. can you suggest me what steps you followed ?

Are you able to share the errors you are seeing? You can follow the getting started guide: https://github.com/meta-llama/llama-stack/blob/main/docs/getting_started.md

I'm also interested to hear how we can run any ollama models?

Curious to understand your use case. Can you share some details?

[heiqs]

Why we cannot run orher models like llama3.2 when using ollama? {"message": "400: Invalid value: Could not find provider for Llama3.1-8B-Instruct"}}. Only Llama3.1-8B-Instruct works when using ollama as inference.