search

opensearch-project / job-scheduler

Run periodic jobs on the cluster from your plugin
Apache License 2.0
21 stars 73 forks source link

[Bundle Build Break] Integration tests are run during build workflow #57

Closed peternied closed 2 years ago

peternied commented 3 years ago

Describe the bug During the build workflow, script file here disables integration test with -DskipTests=true but integration tests are still running during the workflow.

Expected behavior Integration tests should be disabled to prevent delays in built time for the OpenSearch distribution. Integration test should run in other workflow steps.

The build script should be updated to have the correct flags, see https://github.com/opensearch-project/opensearch-build/tree/main/bundle-workflow/scripts/components for more details on how distribution builds are triggered and can be modified.

peternied commented 3 years ago

Found when #56 was discovered, see that bug for a full execution logs.

peternied commented 3 years ago

Another build was blocked because of an integration test failure:

REPRODUCE WITH: ./gradlew ':opensearch-job-scheduler-spi:integTest' --tests "org.opensearch.jobscheduler.spi.utils.LockServiceIT.testRenewLock" -Dtests.seed=BD9E621F27BCF716 -Dtests.security.manager=false -Dtests.locale=en-IE -Dtests.timezone=Asia/Kathmandu -Druntime.java=14

org.opensearch.jobscheduler.spi.utils.LockServiceIT > testRenewLock FAILED
    com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=29, name=opensearch[transport_client_external_0][listener][T#1], state=RUNNABLE, group=TGRP-LockServiceIT]
        at __randomizedtesting.SeedInfo.seed([BD9E621F27BCF716:184EAC40754E36A7]:0)

        Caused by:
        java.lang.AssertionError: Failed to release lock.
            at __randomizedtesting.SeedInfo.seed([BD9E621F27BCF716]:0)
            at org.junit.Assert.fail(Assert.java:88)
            at org.junit.Assert.assertTrue(Assert.java:41)
            at org.opensearch.jobscheduler.spi.utils.LockServiceIT.lambda$testRenewLock$63(LockServiceIT.java:488)
bowenlan-amzn commented 2 years ago

Using assemble instead of build https://github.com/opensearch-project/job-scheduler/blob/main/scripts/build.sh#L75

peternied commented 2 years ago

The root cause of this issue is that a failed build can hold up the organization, we should not signal a failure during the opensearch-build process for a distribution build if its a flaky test case.

@bowenlan-amzn The flaky test case does not seem to be addressed, has that been resolved in another issue or pull request?

bowenlan-amzn commented 2 years ago

I thought assemble resolves this issue, unless there's some concern of using it other than gradle build without running test tasks.

For not running test during ./gradlew build, the special thing in this repo I just checked is we have a test task called javaRestTest, it's actually similar to integTest in other plugin repo. ./gradlew build -x integTest -x javaRestTest -x test should disable all tests.

And yeah, you created a issue for the flaky test https://github.com/opensearch-project/job-scheduler/issues/56 😛, but we haven't got time to debug it.

peternied commented 2 years ago

Thanks @bowenlan-amzn, sounds like #56 is the only follow up that is needed. Closing this issue out.