local llm

[zhanpengjie]

how to set base_url and model in python sdk?

[pradhyumna85]

@zhanpengjie which local model you are trying to use? Does that have vision capability?

[torrischen]

I also have problem with using llama3.2-90B-Vision with vllm. The error said environment variable missing?

[pradhyumna85]

@torrischen refer the model and api_base params here and pass it accordingly in zerox: https://docs.litellm.ai/docs/providers/vllm

Also refer #65

[torrischen]

@torrischen refer the model and api_base params here and pass it accordingly in zerox: https://docs.litellm.ai/docs/providers/vllm

Also refer #65

Thanks. That’s helpful