Improving GitHub Issue Management for CNCF projects using LLMs

[rajaskakodkar]

Currently, every GitHub project and specially the ones that come under CNCF use independent processes for issue triage, bot replies and so on. At a broad level, the following patterns arise where project maintainers have to handle chop wood carry water tasks.

• Detecting duplicate issues
• Identifying potential fixes/resolution for the issues which are trivial
• Setting up machinery for bot replies for community engagement

With the use of AI models, running on cloud native infra, the above tasks can be automated using

• Using generative models(LLMs) to provide actionable outputs based on issue description and predictive models to group issues or help in triaging
• Fine tuning of models based on the data of issues/PRs/repository of a specific project
• Providing a well defined user experience for maintainers to set up processes for projects and tweaking, fine tuning LLMs
• Providing required hooks to course correct models in case of false positives and false negatives

This can benefit with:

• Providing bug fixes based on the issue description
• Detecting spam
• Having the ability to interact with an issue/PR to help with debugging, asking for reviews, next steps and so on

The following falls in the purview of this issue:

• Building the machinery of identifying models, fine tuning them and deploying/serving them
• Identifying a project under CNCF org to try out this approach
• Possibly moving the project for developing this tool under https://github.com/cncf-tags/cloud-native-ai

[rajaskakodkar]

cc @raravena80 @cathyhongzhang @zanetworker @ronaldpetty @rootfs

[nikhita]

cc @dynamicwebpaige -- since we talked about this at KubeCon and you had ideas around this

[loganloganlogan]

rajaskakodkar Are you familiar with https://dosu.dev/? It's free for Foundation-backed OSS and does some of what you're suggesting. devstein is the person to talk to.

[devstein]

Thanks for the ping @loganloganlogan!

Hey @rajaskakodkar 👋 at Dosu we are building exactly what you described! Dosu helps with auto-labelling, issue deduplication, spam detection, question answering, and bug triage.

We've already helped resolve thousands of issues across thousands of projects like Apache Superset, Element/Matrix, LlamaIndex, and more.

We worked with @krook to give a special free tier for all CNCF projects (tweet). Would love to chat more it.

[raravena80]

Sounds exciting. Hoping to see this integrated here soon!