search

princeton-nlp / SWE-agent

SWE-agent takes a GitHub issue and tries to automatically fix it, using GPT-4, or your LM of choice. It solves 12.47% of bugs in the SWE-bench evaluation set and takes just 1 minute to run.
https://princeton-nlp.github.io/SWE-agent/
MIT License
12.12k stars 1.21k forks source link

Support Deepinfra API #266

Closed EwoutH closed 2 months ago

EwoutH commented 3 months ago

Describe the feature

Deepinfra is currently one of the cheapest API providers of LLM inference (together with groq see #108), which makes it interesting to support their API. They currently have the cheapest OpenChat, mixtral and Llama 3 inference token pricing available.

Having more API providers supported also decreases dependability on any particular one.

Potential Solutions

Support the Deepinfra API. See their docs.

AxDSan commented 3 months ago

I've seen an increased number of people interested in adding more and more different kinds of providers (myself included #261 , and the list can get so large, I am voting in favor of the implementation for LiteLLM which supports the vast majority of all of these providers including the legacy ones (OpenAI, Anthropic, to mention a few)...

LiteLLM does removes the verbosity behind the conundrum of adding more and more providers by supporting them right out-of-the-box, which like I expressed, I feel it would make this particular process a bit more tedious if not more complex due to the different factors of everything affecting the provider selections, adding models, etc., etc.,

Given the way SWE-Agent is growing with the interest of people wanting to add more providers, I propose that LiteLLM should be integrated with SWE-Agent so that way we just have one main chain of doing things, instead of focusing of adding all of these providers one by one, and why reinvent the wheel when there's a solution already that can be adopted?

klieret commented 3 months ago

I agree with this. I'd be open to add litellm, ideally starting with some of the less important or newer models (anything but gpt4, claude 2, claude 3). Basically I'd want to move slowly so we don't disrupt the research that uses those 3 models. Please keep in mind that a lot of the cheaper models are too bad to really be used for swe-agent.

ofirpress commented 2 months ago

We're going to support something like LiteLLM which will deprecate the need for this issue. Closing.