Check changed files before running gradle check

[gaiksaya]

Description

The repository requires the gradle check job status to pass before merging any PR. With path-ignore filter, this status check was skipped. However, GitHub does not resolve this conflict or allows to specify the requirement to be optional. See https://github.com/opensearch-project/OpenSearch/pull/13494 still waiting for gradle-check status to be set. There are multiple discussions on GH community forums https://github.com/orgs/community/discussions/44490 and all of them requires work around as of now. This change adds one more step to gradle-check job to detect the files changed and accordingly decide whether to run the consecutive steps or not. See examples below:

1. Where gradle check runs for change in build.gradle https://github.com/gaiksaya/OpenSearch/pull/10
2. Gradle check steps are skipped for changes in md file https://github.com/gaiksaya/OpenSearch/pull/11

I set my fork requirements to mimic gradle check status as required and this satisfies both conditions

Related Issues

Related https://github.com/opensearch-project/OpenSearch/issues/4053

Check List

• [x] New functionality includes testing.
  • [x] All tests pass
• [x] New functionality has been documented.
  • [x] New functionality has javadoc added
• [x] Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
• [x] Commits are signed per the DCO using --signoff
• [x] Commit changes are listed out in CHANGELOG.md file (See: Changelog)
• [x] Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions[bot]]

:x: Gradle check result for 6179f010edbd010488a8db0df3ee52c6d3b0f493: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 574d7184f70c8a43dfd5714c0624ddf8a6f1a065: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:white_check_mark: Gradle check result for 6179f010edbd010488a8db0df3ee52c6d3b0f493: SUCCESS

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 71.49%. Comparing base (b15cb0c) to head (6179f01). Report is 307 commits behind head on main.

:exclamation: Current head 6179f01 differs from pull request most recent head 0de7d21

Please upload reports for the commit 0de7d21 to get more accurate results.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #13498 +/- ## ============================================ + Coverage 71.42% 71.49% +0.07% - Complexity 59978 61012 +1034 ============================================ Files 4985 5050 +65 Lines 282275 286806 +4531 Branches 40946 41552 +606 ============================================ + Hits 201603 205046 +3443 - Misses 63999 64848 +849 - Partials 16673 16912 +239 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[gaiksaya]

@kotwanikunal @reta Can you please review this change? Let me know if I need to add more maintainers. Thanks!

[reta]

@gaiksaya @dblock have some thoughts regarding the problem over the weekend, I think the idea is sound but the way we approach it probably not. Primarily, because for some reason, we only focus on Gradle check but the same "skip me" rules apply to other workflows like assemble (and very likely even precommit).

What if we combine changeset detection + Gradle build instead, and just skip most of the tasks if there are no relevant changes?

[dblock]

What if we combine changeset detection + Gradle build instead, and just skip most of the tasks if there are no relevant changes?

Maybe.

I am confused why we needed to do anything special about gradle check in terms of requiring it to pass. Isn't that what any check does by default?

[reta]

I am confused why we needed to do anything special about gradle check in terms of requiring it to pass. Isn't that what any check does by default?

I mean - we don't need to run the whole test suite if only CHANGELOG.md changes, right?

[gaiksaya]

What if we combine changeset detection + Gradle build instead, and just skip most of the tasks if there are no relevant changes?

Maybe.

I am confused why we needed to do anything special about gradle check in terms of requiring it to pass. Isn't that what any check does by default?

Other workflows are not set to mandatory at branch settings. Current settings for this repo that applies to all branches:

[gaiksaya]

@gaiksaya @dblock have some thoughts regarding the problem over the weekend, I think the idea is sound but the way we approach it probably not. Primarily, because for some reason, we only focus on Gradle check but the same "skip me" rules apply to other workflows like assemble (and very likely even precommit).

What if we combine changeset detection + Gradle build instead, and just skip most of the tasks if there are no relevant changes?

I believe main reason would be other workflows are not as flaky and not so time consuming. Gradle check is a whole different story. What you are suggesting does not need a common workflow. Just adding path-ignore in those workflows would work as GH would just skip them and won't wait at PR merge asking for status. Drawback: Each workflow would have it own list to ignore which I think is fine?

Also, I believe you mean gradle tasks here? just skip most of the tasks That would need to break down the gradle check right?

[reta]

Thanks @gaiksaya

Drawback: Each workflow would have it own list to ignore which I think is fine?

I think the ignore list is per repository, right? The only thing we may need to figure out - what has changed (changeset)

Also, I believe you mean gradle tasks here? just skip most of the tasks That would need to break down the gradle check right?

Correct, more like onlyIf { hasChanges() } condition.

[gaiksaya]

I think the ignore list is per repository, right? The only thing we may need to figure out - what has changed (changeset)

It's per workflow. Example: https://github.com/opensearch-project/OpenSearch/pull/13477/files# and path.ignore

Also, I believe you mean gradle tasks here? just skip most of the tasks That would need to break down the gradle check right?

Correct, more like onlyIf { hasChanges() } condition.

Can this be a part of gradle check breakdown issue?

[reta]

It's per workflow. Example: https://github.com/opensearch-project/OpenSearch/pull/13477/files# and path.ignore

This is how we implemented it, correct, but it does not need to be

Can this be a part of gradle check breakdown issue?

May be, I just wanted to run the idea over you folks, I haven't prototyped it

[dblock]

I think we should decide whether we want to merge this as is. WDYT @gaiksaya @reta? I am good with whatever you two choose!

[gaiksaya]

I think we should decide whether we want to merge this as is. WDYT @gaiksaya @reta? I am good with whatever you two choose!

I am good to merge as is too.

[reta]

I am good to merge as is too.

We have no better working option at the moment (and @gaiksaya did really good job here figuring out GA internals) but do we need to rush the merge? The workflow becomes difficult to follow and change, this is the concern I have (and that is the motivation to look for other options).

[github-actions[bot]]

:x: Gradle check result for 1e3fed2dc6804e1517f37e971948c69e014480e5: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 0234aa7aae725d5a95bcb0a29fbf2295810ca5a3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:x: Gradle check result for 0234aa7aae725d5a95bcb0a29fbf2295810ca5a3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[gaiksaya]

Looks like we might need to update the required check after this is merged, @gaiksaya can you watch/resolve any issues that come up post merge?

Will do.

[github-actions[bot]]

:x: Gradle check result for 0234aa7aae725d5a95bcb0a29fbf2295810ca5a3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions[bot]]

:white_check_mark: Gradle check result for 0de7d2163f9d71baf556c033a549e7ab396cefb9: SUCCESS

[gaiksaya]

Sorry @peternied @reta Need approval again in order for minimal approval count workflow to pass. I will be merging the PR after long weekend to be available for surprise failures :)

[gaiksaya]

I added backporting to 2.x label. Any more branches that this PR should be backported to? Changing the required workflow to pass check from gradle-check to check-result

[reta]

I added backporting to 2.x label. Any more branches that this PR should be backported to?

I think 1.x / 1.3 would benefit from that as well, thank you.

[gaiksaya]

I added backporting to 2.x label. Any more branches that this PR should be backported to?

I think 1.x / 1.3 would benefit from that as well, thank you.

Done! Thank you!