[Feature] Branch other LLM to perform the task

[franckleveque]

Hi,

Your article is quite interesting, I was wondering if it would be relatively simple to branch another LLM on the market (like Claude or LLaMa 2) ?

If I understood properly your files , it is https://github.com/Pythagora-io/gpt-pilot/blob/main/pilot/utils/llm_connection.py that is in charge managing the connection. Are there other part that takes care of the communication with the LLM model ?

[zvone187]

Thank you 🙏

Yes, you're right. There, you can just change the endpoint for the LLM. Unfortunately, we've used OpenAIs function calling which won't work with other LLMs so it will require a bigger refactor. The good thing is that function calling needs to be removed anyway since it doesn't work too good anyway.

[mrgoonie]

I've created this PR with OpenRouter API so we can use other LLM models

[LeonOstrez]

now you are able to set in .env either OPENAI or AZURE or OPENROUTER