JACoB PR for Issue `rewriteIssue` Function Generates Incorrect and Verbose Outputs

[jacob-ai-bot[bot]]

Summary:

Description

The rewriteIssue function in the githubRouter TRPC route on the backend is malfunctioning by referencing nonexistent file names and producing overly verbose outputs. This leads to the generation of issues with inaccurate information and unnecessary details, hindering the development workflow.

Expected Behavior

• The rewriteIssue function should generate issues that accurately reflect only the existing files in the system.
• The output should be concise, containing relevant information without unnecessary verbosity.

Actual Behavior

• The function references file names that do not exist in the system, indicating it is "hallucinating" file names.
• The output includes excessive information about adding tests and other irrelevant data, making the issues unnecessarily wordy.

Additional Information

• It is crucial to update the TRPC route to ensure that the rewriteIssue function:
  • Uses only the actual file names from the codebase context to prevent hallucinations.
  • Generates concise and precise issues that do not require user modification.
  • Includes an overview of every file in the codebase to maintain consistency and accuracy in issue generation.

Proposed Solution

• Refactor the rewriteIssue function to strictly reference existing files by passing codebase context with overviews of each file in the codebase.
• Pass in the repo and org so the function can get the codebase context. See the evaluateIssue function in the githubRouter file to get an example of how that is done. Refactor that out into a shared function that can take a repoOwner, repoName, and accessToken, and return the codebase context for that project.
• Update the prompt to simplify the output generation logic to eliminate unnecessary verbosity and ensure clarity in the issues produced.

@jacob-ai-bot --skip-build

Plan:

Step 1: Edit /src/server/api/routers/github.ts

Task: Add org and repo parameters to rewriteIssue function

Instructions: Modify the rewriteIssue function in /src/server/api/routers/github.ts to accept org and repo as input parameters. Ensure these parameters are used in the function's logic to retrieve the codebase context.

Exit Criteria: The rewriteIssue function accepts org and repo parameters and uses them appropriately in its logic.

Step 2: Edit /src/server/api/routers/github.ts

Task: Refactor evaluateIssue to use shared codebase context function

Instructions: Modify the evaluateIssue function in /src/server/api/routers/github.ts to use the new shared utility function from /src/server/api/utils.ts to retrieve the codebase context, ensuring consistent behavior across functions.

Exit Criteria: evaluateIssue function uses the shared utility function to retrieve codebase context without any loss of existing functionality.

Step 3: Edit /src/server/api/routers/github.ts

Task: Update rewriteIssue to include codebase context in AI prompt

Instructions: Modify the rewriteIssue function in /src/server/api/routers/github.ts to use the shared utility function to retrieve the codebase context. Include the codebase context in the prompt sent to the AI model to ensure it references existing files only.

Exit Criteria: rewriteIssue function includes codebase context in its prompt to the AI model and no longer references nonexistent files.

Step 4: Edit /src/server/api/routers/github.ts

Task: Revise AI prompt in rewriteIssue to improve output

Instructions: Update the prompt used in the rewriteIssue function to instruct the AI model to reference only existing files from the codebase context and generate concise issue descriptions without unnecessary verbosity.

Exit Criteria: The AI prompt is updated; rewriteIssue generates concise and accurate issue descriptions referencing only existing files, and the output does not include irrelevant or verbose information.

Step 5: Edit /src/server/api/utils.ts

Task: Implement shared function to retrieve codebase context

Instructions: Create a new function in /src/server/api/utils.ts that accepts repoOwner, repoName, and accessToken, and returns the codebase context for the specified repository. Extract and refactor the relevant code from the evaluateIssue function to avoid duplication.

Exit Criteria: A new utility function exists in /src/server/api/utils.ts that returns codebase context given repoOwner, repoName, and accessToken.

[jacob-ai-bot[bot]]

Hello human! 👋

This PR was created by JACoB to address the issue rewriteIssue Function Generates Incorrect and Verbose Outputs

Next Steps

1. Please review the PR carefully. Auto-generated code can and will contain subtle bugs and mistakes.

2. If you identify code that needs to be changed, please reject the PR with a specific reason. Be as detailed as possible in your comments. JACoB will take these comments, make changes to the code and push up changes. Please note that this process will take a few minutes.

3. Once the code looks good, approve the PR and merge the code.