[Bug]: OpenSearch Dashboards Integration tests did not pick up `latest` build number of OpenSearch

[prudhvigodithi]

Describe the bug

When an OpenSearch and OpenSearch Dashboards build started, there is a parameter UPDATE_LATEST_URL that is check-boxed, this UPDATE_LATEST_URL is used in URL's example https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/2.16.0/latest/linux/x64/tar/builds/opensearch/manifest.yml to download the latest successful version specific built artifact. The same latest is used for OpenSearch Dashboards Integration tests to downloads the latest successful built OpenSearch artifact. Example.

This setup had no issues and is able to download the right latest successful built artifact and used for running the tests. But for 2.16.0 integration tests specifically for RC7 the latest was pointed to 10140 (one before the latest successful build 10144 for 2.16.0) which should be 10144. Example build: https://build.ci.opensearch.org/job/integ-test-opensearch-dashboards/6188/

This dint happen for RC6 and RC5, the latest point to right build right

For RC6

<img width="633" alt="Screenshot 2024-08-04 at 10 05 42 AM" src="https://github.com/user-attachments/assets/8bf9d572-3fcd-4

For RC5

716-9b06-2f9839907909">

To reproduce

Not reproducible as this is flaky issue at the cloud front level used for https://ci.opensearch.org/ci/dbc/. But moving forward for RC (release candidate) specific integration tests use the specific build number rather using the latest to infer the build number, for daily builds we can continue to use the latest.

Expected behavior

No response

Screenshots

If applicable, add screenshots to help explain your problem.

Host / Environment

No response

Additional context

No response

Relevant log output

No response

[gaiksaya]

[Triage]

We can separate out RC build criteria from regular builds. Will create another issue for that.