Implement GitHub Comments Benchmark Stats Collection and Storage

[cshezi]

Description: Set up a system to collect benchmark statistics from GitHub comments, store them in a CSV file within a designated repository, and use this repository as a persistent storage for easy trend analysis and A/B comparison in pull requests.

Tasks:

1. Benchmark Stats Extraction:

   • Implement a script to extract benchmark statistics from GitHub comments. Utilize GitHub API for retrieving comments on relevant issues or pull requests.

2. CSV Storage:

   • Create a CSV file structure to store benchmark statistics. Include columns for relevant information such as processor details, benchmark values, and timestamps.

3. Repository Setup:

   • Establish a repository for storing benchmark CSV files. Ensure appropriate access controls are set.

4. Automation:

   • Set up a workflow that will be triggered when the PR is merged in the main branch which will store or append the final benchmark stats of the incoming changes

5. Documentation:

   • Provide clear documentation on how developers can interpret and contribute to the benchmark stats collection process.

6. Enhance PR Comments:

   • Implement a feature to automatically enhance pull request comments with A/B comparison benchmark stats. This can be achieved by referencing the CSV file in the repository.

Data to Consider for storing Benchmark stats: Max Response time PR: Author PR: Merge time and date

Additional Considerations:

• Ensure privacy and security measures are in place when dealing with benchmark stats, especially if the repository is public.
• Consider versioning in the CSV file or separate files for different processors to facilitate efficient trend analysis.

Deadline: 4 days max : 0.5 days Research, 0.5 days Documentation, 3 days Implementation