Mechanism to close the created Gradle Check AUTOCUT flaky test issues

[prudhvigodithi]

Description

• Updated the logic to query the cluster for flaky tests with user defined timeFrame rather that default to an monthly created index. This can be updated dynamically without the library release using timeFrame arg.

• New library gradleCheckFlakyTestGitHubIssue to handle the issue creation, edit issue body and skip when issue is closed. This library is inspired from createGithubIssue but used only by gradleCheckFlakyTestDetector with some additional logic used for only gradle check flaky issue creation automation.

• New groovy class MarkdownComparator to compare two markdown tables in the format created by class CreateMarkDownTable. This uses the row part of the markdown table and identifies if there is any difference found between the data part of the OpenSearch metrics cluster and part of the created issue body. If there is no difference the gradleCheckFlakyTestGitHubIssue wont re-open the issue closed by the user. If found difference the gradleCheckFlakyTestGitHubIssue will re-open and update the issue body with the new data.

Differences found:
Git Reference: f8213b8492b213922bea35d22c4317a79786a74f, Merged Pull Request: [14357](https://github.com/opensearch-project/OpenSearch/pull/14357), Build Details: [41268](https://build.ci.opensearch.org/job/gradle-check/41268/testReport/), Test Name: `org.opensearch.indices.IndicesRequestCacheIT.testCacheCleanupWithDefaultSettings {p0={"search.concurrent_segment_search.enabled":"false"}}`

• New class ParseMarkDownTable is added to just parse the markdown tables in the format created by class CreateMarkDownTable. The ParseMarkDownTable is used for parsing the issue body and for the generated test report from data from Metrics cluster.

• Added token: ${{ secrets.CODECOV_TOKEN }} to fix the error

  [2024-06-25T00:24:29.694Z] ['error'] There was an error running the uploader: Error uploading to [https://codecov.io:](https://codecov.io/) Error: There was an error fetching the storage URL during POST: 429 - {'detail': ErrorDetail(string='Rate limit reached. Please upload with the Codecov repository upload token to resolve issue. Expected time to availability: 1559s.', code='throttled')}
  [2024-06-25T00:24:29.695Z] ['info'] Codecov will exit with status code 0. If you are expecting a non-zero exit code, please pass in the `-Z` flag

Issues Resolved

Part of https://github.com/opensearch-project/OpenSearch/issues/14475

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 77.84810% with 35 lines in your changes missing coverage. Please review.

Project coverage is 84.23%. Comparing base (cfba629) to head (e5176cf). Report is 3 commits behind head on main.

Files	Patch %	Lines
tests/gradlecheck/MarkdownComparatorTest.groovy	75.00%	0 Missing and 31 partials :warning:
src/gradlecheck/MarkdownComparator.groovy	55.55%	0 Missing and 4 partials :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #448 +/- ## ============================================ - Coverage 87.12% 84.23% -2.90% - Complexity 31 80 +49 ============================================ Files 88 105 +17 Lines 233 520 +287 Branches 12 61 +49 ============================================ + Hits 203 438 +235 - Misses 22 26 +4 - Partials 8 56 +48 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[prudhvigodithi]

The reason the codecov/patch — 77.84% of diff hit (target 87.12%) is failed because the target: auto so compares the percentage with the current head commit. Changed to target: 70% ( From OpenSearch and build repo). Overall with this PR the main branch codeCov will be 84.23%. Thank you

[rishabh6788]

Looks fine overall, didn't really get the comparison logic. Can you help explain how it will handle closed PRs, e.g., in https://github.com/opensearch-project/OpenSearch/issues/14472 the PR mentioned in the table was closed and the change was merged in a different PR. Is this case also handled?

[prudhvigodithi]

Hey @rishabh6788 the logic assumes that the AUTOCUT issue created by the automation is fixed by the assigned owner, who then closes the issue. If the automation identifies a flaky test again (e.g., within the last 30 days and after the fix), and the issue is closed, the automation will compare the issue body table for all tests against the actual test results from the cluster query. If they match, the issue remains closed. However, if the query results differ (e.g., different post-merge commit or test results), the automation will reopen the issue and update the issue body with the new failing test data and linking the pull requests. The automation wont compare the open/closed status of past or present pull requests, once the issue is fixed and the pull requests are re-based, the flaky error should no longer be present. If the flaky error persists, the user can reopen the issue, and the automation will continue to update the issue body and link back the pull requests. Thank you @gaiksaya @getsaurabh02

[prudhvigodithi]

I believe this will need version bump?

Ya let me update the version Sayali.

[prudhvigodithi]

Thanks @gaiksaya and @rishabh6788 I will go ahead and merge this PR.