Closed monusingh-1 closed 11 months ago
Hi @monusingh-1
The builds are not flaky but the way GH create issue is written is buggy. As per my analysis this is the culprit: https://github.com/opensearch-project/opensearch-build/blob/main/manifests/3.0.0/opensearch-3.0.0.yml#L9 https://github.com/opensearch-project/opensearch-build/blob/main/jenkins/opensearch/distribution-build.jenkinsfile#L645
Looks like our infra is building on different java versions for linux and windows. Since plugins did not upgrade to jdk 20 everything fails on linux which is legit. However, (I think and pretty sure) on windows it finds compatible java version and builds itself successfully closing all the GH issues for all below components that are and should be broken today:
++ scoop list jdk
++ grep 20
++ head -1
++ cut -d ' ' -f1
+ scoop reset openjdk14
Resetting openjdk14 (14.0.2-12).
2023-10-17 13:11:57 ERROR Failed plugins are ['opensearch-reports', 'performance-analyzer', 'opensearch-observability', 'k-NN', 'neural-search', 'geospatial', 'cross-cluster-replication', 'sql', 'asynchronous-search']
@peterzhuamazon created a PR to fix these recently. However, the failures are right. The error mentioned by you is java incompatible error (have seen that before). Thanks!
I think the above PR fixes the false positive. 3.0.0 is building right now https://build.ci.opensearch.org/blue/organizations/jenkins/distribution-build-opensearch/detail/distribution-build-opensearch/8677/pipeline/75 and I can see geospatial plugin failing on windows similar to linux (it was not happening before). Windows is also fixed now:
2023-10-19 01:11:09 ERROR Failed plugins are ['opensearch-reports', 'opensearch-observability', 'k-NN', 'neural-search', 'geospatial', 'cross-cluster-replication', 'sql', 'asynchronous-search']
Will create an issue to fix the GH creation logic. Thanks!
Created issue to add more details to GH issue https://github.com/opensearch-project/opensearch-build/issues/4155 Closing this issue.
Describe the bug
When OS builds are triggered the workflow fails intermittently and cuts issue on plugin repos, the issues are then resolved in next run. When the next run happens a new issue is created.
129 issues have been cut so far https://github.com/opensearch-project/cross-cluster-replication/issues?q=is%3Aissue++%5BAUTOCUT%5D+Distribution+Build+Failed+for+cross-cluster-replication-3.0.0+
https://build.ci.opensearch.org/job/distribution-build-opensearch/8674/consoleFull On CCR
on K-nn
Could you please confirm if the issues are actionable by plugin maintainers ? How can we address the falky nature of this build ?
To reproduce
-
Expected behavior
No response
Screenshots
If applicable, add screenshots to help explain your problem.
Host / Environment
No response
Additional context
No response
Relevant log output
No response