I found that this test is flakily fails with the below assertion failure. Hence, I suggest a new way to fix the test by adding some synchronization for the test execution only. I at first identify the source code location whose slow execution leads to the flaky test failure, where if org/java_websocket/WebSocketImpl.java#572 slows to set the value of readyState then the test fails (org.java_websocket.issues.Issue677Test#119). Hence, I introduce one variable in this class [WebSocketImpl.java] that is only there to provide some synchronization. By analyzing the code, I found if the assertion check happens before the 2times execution of this line, then the assertion is failing. Hence, until this statement is executed two times, I force the thread that the test runs on to wait before it accesses the value of webSocket0.isClosed(). The waiting location is at before org.java_websocket.issues.Issue677Test#119.
Related Issue
Running org.java_websocket.issues.Issue677Test
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.286 sec <<< FAILURE!
testIssue(org.java_websocket.issues.Issue677Test) Time elapsed: 0.265 sec <<< FAILURE!
java.lang.AssertionError: webSocket.isClosed()
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.java_websocket.issues.Issue677Test.testIssue(Issue677Test.java:119)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:242)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:137)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Description
I found that this test is flakily fails with the below assertion failure. Hence, I suggest a new way to fix the test by adding some synchronization for the test execution only. I at first identify the source code location whose slow execution leads to the flaky test failure, where if org/java_websocket/WebSocketImpl.java#572 slows to set the value of readyState then the test fails (org.java_websocket.issues.Issue677Test#119). Hence, I introduce one variable in this class [WebSocketImpl.java] that is only there to provide some synchronization. By analyzing the code, I found if the assertion check happens before the 2times execution of this line, then the assertion is failing. Hence, until this statement is executed two times, I force the thread that the test runs on to wait before it accesses the value of webSocket0.isClosed(). The waiting location is at before org.java_websocket.issues.Issue677Test#119.
Related Issue
Running org.java_websocket.issues.Issue677Test Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.286 sec <<< FAILURE! testIssue(org.java_websocket.issues.Issue677Test) Time elapsed: 0.265 sec <<< FAILURE! java.lang.AssertionError: webSocket.isClosed() at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.java_websocket.issues.Issue677Test.testIssue(Issue677Test.java:119) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:242) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:137) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Results :
Failed tests: testIssue(org.java_websocket.issues.Issue677Test): webSocket.isClosed()
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
Motivation and Context
The test is flaky, and it is passing and failing in different runs.
How Has This Been Tested?
I run this test 1000 times on Ubuntu 20.04 with Java 11 and it always passes.
Types of changes
Checklist: