Enhance the Jira to GitHub issue conversion

kleneway commented 13 hours ago

Summary

Many incoming Jira tickets lack sufficient detail for JACoB to process them effectively. This leads to wasted resources as we create plans and research items for tasks that cannot be completed due to inadequate information. To improve efficiency, we need to implement an evaluation step before converting Jira issues into GitHub issues.

Proposed Solution

Enhance the Jira to GitHub issue conversion process by introducing an issue quality evaluation step. Instead of always rewriting the Jira title and description and creating a new GitHub issue, we will first assess the quality of the Jira issue to determine if it contains enough detail for JACoB to handle.

Implementation Steps

1. Implement `evaluateJiraIssue` Function

File: /src/server/utils/evaluateIssue.ts
Task:
- Add a new function evaluateJiraIssue that:
- Accepts the title and description of a Jira issue as inputs.
- Returns an evaluation score between 1 and 5 (half-points acceptable), indicating how likely it is that an AI coding agent can flawlessly complete the task.
- Provides a one-sentence feedback message if the score is less than 4, informing the user what needs to be changed to make the ticket actionable by JACoB.
- Note: Define clear evaluation criteria for scoring to ensure consistent assessments.

2. Update the Jira Issue Conversion Process

File: /src/server/utils/jira.ts
Task:
- Modify the convertJiraIssueToGithubIssue function to include the evaluation step:
- Before creating a GitHub issue, call the evaluateJiraIssue function with the Jira issue's title and description.
- If the evaluation score is less than 4:
  - Do not create a GitHub issue.
  - Save the evaluation score, feedback message, and Jira issue description in the database.
- If the evaluation score is 4 or higher:
  - Proceed with rewriting the Jira issue and creating a GitHub issue as usual.

3. Update the Issues Database Schema

File: Create migration script /src/server/db/migrations/20241130000000_updateIssuesTable.ts
Task:
- Modify the issues table to include the following optional fields:
- jiraIssueDescription: text field to store the original description from the Jira issue.
- evaluationScore: numeric field to store the score from evaluateJiraIssue.
- feedback: text field to store the feedback message provided to the user.
- didCreateGithubIssue: boolean field (default false) indicating whether a GitHub issue was created.
- Ensure the migration handles existing data appropriately to maintain data integrity.

4. Communicate Feedback to Users

Task:
- User Interface Update:
- Update the UI components in the Jira integration (e.g., issue status displays) to show the feedback message when a Jira issue is not converted.
- Alternative Notification:
- Optionally, implement a feature to send a notification or comment directly on the Jira issue with the feedback message.
- Note: Clearly specify in documentation or code comments how the feedback is communicated to users.

5. Update Tests and Documentation

Task:
- Testing:
- Update existing unit tests or create new tests to cover:
  - The evaluateJiraIssue function for correct scoring and messaging.
  - The updated convertJiraIssueToGithubIssue function for both paths (issue created and not created).
- Documentation:
- Update relevant documentation to reflect the changes made to the issue conversion process.
- Include guidelines on the evaluation criteria used by evaluateJiraIssue.

Expected Outcome

By implementing this evaluation step in the Jira to GitHub issue conversion process:

Improved Efficiency: JACoB will focus on Jira tickets that have sufficient detail, improving resource utilization.
Resource Optimization: Resources will no longer be wasted on creating plans and research items for incomplete or vague tickets.
User Feedback: Users will receive immediate feedback on how to improve their Jira tickets to make them actionable by JACoB.
Enhanced AI Effectiveness: The overall effectiveness of the AI coding agent will be enhanced as it works with well-defined tasks. @jacob-ai-bot --skip-build

jacob-local-kevin[bot] commented 13 hours ago

JACoB here...

You mentioned me on this issue and I am busy taking a look at it.

I'll continue to comment on this issue with status as I make progress.