Current JACoB uses a runtime-generated source map to identify the file names, types, and other important information for providing context of what files should be updated for a given issue. This actually works surprising well for many issues, however it falls short for more complex, multi-step bug fixes.
I previously implemented a very simple RAG approach last year which didn't work very well. One big challenge was that the semantic similarity of a GitHub issue and a code file are very different. Instead we need to convert the file into a format that more closely matches the type of information found in an issue to better identify the correct files to update.
This is a critical step in making JACoB work so we also want to use this approach as part of the user chat to identify the potential files to update and ask the user to confirm that this file/files are correct before starting work on the issue.
Here are some steps to implement the RAG system.
[ ] Update the JACoB postgres database to handle vector embeddings
[ ] Add a new table to store the embeddings on a per-project basis
[ ] Use a fast/cheap LLM (Gemini Flash or GPT-4o) to write a 1-2 sentence description of each file in the codebase
[ ] Create a vector embedding for the descriptions, along with metadata about the file
[ ] Use a standard RAG retrieval & ranking algorithm to identify the files that are most likely to need to be changed for the given plan step
[ ] Provide these files & descriptions as additional context to the extendedIssue prompt
Current JACoB uses a runtime-generated source map to identify the file names, types, and other important information for providing context of what files should be updated for a given issue. This actually works surprising well for many issues, however it falls short for more complex, multi-step bug fixes.
I previously implemented a very simple RAG approach last year which didn't work very well. One big challenge was that the semantic similarity of a GitHub issue and a code file are very different. Instead we need to convert the file into a format that more closely matches the type of information found in an issue to better identify the correct files to update.
This is a critical step in making JACoB work so we also want to use this approach as part of the user chat to identify the potential files to update and ask the user to confirm that this file/files are correct before starting work on the issue.
Here are some steps to implement the RAG system.
extendedIssueprompt