How could I use local model (not any api)

[hgcdanniel]

how could I call the local model? The format is Huggingface, but the model is not deployed to the inference endpoint and there is no api_base. I just want to use the local model, although I modified model="local/model_name" model_turbo="local/model_name" fallback_models=["local/model_name"] here, but still calls the Huggingface api, I don't want to get huggingface api, thanks a lot.

[mrT23]

this is what we support: https://pr-agent-docs.codium.ai/usage-guide/changing_a_model/

[hgcdanniel]

Thank you very much. This change model link is very amazing, I tried to modify pr_agent/settings/configuration.toml to be my local model according to the Custom Models settings method. but there were still errors. I modified the configuration and init settings, where else should I modify? [pr_agent/settings/configuration.toml] model="/data/xxx/CodeLlama-34b-Instruct-hf" model_turbo="/dataxxx/CodeLlama-34b-Instruct-hf" fallback_models=["/data/xxx/CodeLlama-34b-Instruct-hf"] [pr_agent/algo/init.py] '/data/xxx/CodeLlama-34b-Instruct-hf':4096, the error is : litellm.exceptions.BadRequestError: litellm.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to /data/xxx/CodeLlama-34b-Instruct-hf Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://dviders

[mrT23]

models need to be hosted just stating local model weights is not supported (anywhere). its not real way to deploy.

CodeLlama-34b can be deployed locally via ollama, for examples https://ollama.com/library

Read more about deploying models, and follow guides.