Implement Diff File Processing and Completion Retrieval in Golang

kmesiab / orion

Orion leverages advanced Large Language Models (LLMs) to deliver insightful, detailed code analyses and suggestions 🧠

2 stars 2 forks source link

Implement Diff File Processing and Completion Retrieval in Golang #31

Open kmesiab opened 8 months ago

kmesiab commented 8 months ago

Description

Implement feature to process diff files and retrieve completions from ChatGPT in Golang, integrating results with Report Viewer React component.

Tasks

Implement API integration to send diff files to ChatGPT.
Retrieve and handle completion results from ChatGPT.
Integrate completions with Report Viewer React component.
Conduct thorough testing for functionality and integration.

Acceptance Criteria

Successfully send diff files to ChatGPT.
Accurate retrieval and integration of ChatGPT completions.
Seamless functionality with Report Viewer React component.

frankie-mur commented 8 months ago

Heya @kmesiab, any updates here? Would be happy to give it a go or help out if needed 😄

kmesiab commented 8 months ago

Hey @frankie-mur , I'm traveling at the moment. Go for it!

frankie-mur commented 8 months ago

Currently encountering a rate limit issue, seems like GPT 3.5 only allows 3 request per minute...I figured okay I will just make one request with all the diff content. This request exceeds the token limit for a request. How did we avoid this previously?

kmesiab commented 8 months ago

Oh interesting. Maybe they have changed something, or perhaps you are finally discovering the limit with a specific larger diff.

We will need to account for this. I was concerned we would find ourselves in this kind of situation (hence the issue ticket for creating a token relief valve back in the Ai-Code-Critic repo).

There are a few options here.

Supporting other text generation providers, like Mixtral, ClaudeAI, Gemeni, etc. This will let the user choose a model with a price point that works for them. (This is the one I've been working on). This can be enabled (it appears) relatively painlessly using this library.
We limit the diff sizes by somehow intelligently filtering out smaller irrelevant changes.
Accept the limitations of each model, and supply good error messaging.
Some combination of the above.
Another solution set you may have.

frankie-mur commented 8 months ago

I did initially start with using lingoose (its nice!) but I thought the rate limit issues was due to the lib so I switched to use the same logic that was present in ai-code-critic but same issue arouse.

I like the first option, as supported multiple generation providers would be great as an end user, I guess my main problem is I do not have GPT+ therefore it is difficult for me to test for anything other then a very small PR.

kmesiab commented 8 months ago

Now that you have implemented configuration, allowing custom models should be relatively easy.