[BUG] CreateIndexIT.testRestartIndexCreationAfterFullClusterRestart intermittent failure

[dblock]

REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.action.admin.indices.create.CreateIndexIT.testRestartIndexCreationAfterFullClusterRestart" -Dtests.seed=B4C5EBD894471D4B -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=pl -Dtests.timezone=US/East-Indiana -Druntime.java=17

org.opensearch.action.admin.indices.create.CreateIndexIT > testRestartIndexCreationAfterFullClusterRestart FAILED
    java.lang.AssertionError: timed out waiting for green state
        at __randomizedtesting.SeedInfo.seed([B4C5EBD894471D4B:CCAD58E520A2C7D2]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureColor(OpenSearchIntegTestCase.java:985)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:924)
        at org.opensearch.test.OpenSearchIntegTestCase.ensureGreen(OpenSearchIntegTestCase.java:913)
        at org.opensearch.action.admin.indices.create.CreateIndexIT.testRestartIndexCreationAfterFullClusterRestart(CreateIndexIT.java:352)

https://github.com/opensearch-project/OpenSearch/pull/2096#issuecomment-1039605783

[aasom143]

Ran test for 500 times, it passed successfully.

./gradlew ':server:internalClusterTest' --tests "org.opensearch.action.admin.indices.create.CreateIndexIT.testRestartIndexCreationAfterFullClusterRestart" -Dtests.seed=B4C5EBD894471D4B -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=pl -Dtests.timezone=US/East-Indiana -Dtests.timeoutSuite=10800000! -Dtests.iters=500

> Configure project :
========================= WARNING =========================
         Backwards compatibility tests are disabled!
See https://github.com/opensearch-project/OpenSearch/issues/4173
===========================================================
=======================================
OpenSearch Build Hamster says Hello!
  Gradle Version        : 8.6
  OS Info               : Mac OS X 14.3.1 (aarch64)
  JDK Version           : 21 (OpenJDK)
  JAVA_HOME             : /Users/guptasom/Library/Java/JavaVirtualMachines/openjdk-21/Contents/Home
  Random Testing Seed   : B4C5EBD894471D4B
  In FIPS 140 mode      : false
=======================================
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.BootstrapForTesting (file:/Users/guptasom/Desktop/OpenSearch/test/framework/build/distributions/framework-3.0.0-SNAPSHOT.jar)
WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.BootstrapForTesting
WARNING: System::setSecurityManager will be removed in a future release

> Task :server:internalClusterTest
WARNING: Using incubator modules: jdk.incubator.vector
Apr 17, 2024 5:10:45 PM sun.util.locale.provider.LocaleProviderAdapter <clinit>
WARNING: COMPAT locale provider will be removed in a future release
<====<============-> 98% EXECUTING [10m 37s]
> :s<============-> 98% EXECUTING [10m 38s]ts completed

WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.gradle.api.internal.tasks.testing.worker.TestWorker (file:/Users/guptasom/.gradle/wrapper/dists/gradle-8.6-all/3mbtmo166bl6vumsh5k2lkq5h/gradle-8.6/lib/plugins/gradle-testing-base-8.6.jar)
WARNING: Please consider reporting this to the maintainers of org.gradle.api.internal.tasks.testing.worker.TestWorker
WARNING: System::setSecurityManager will be removed in a future release