Describe the bug
I ran this test 10,000 times in normal settings and got the following failures for many times.
Debug log
Running org.java_websocket.issues.Issue677Test
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.125 sec <<< FAILURE!
testIssue(org.java_websocket.issues.Issue677Test) Time elapsed: 0.103 sec <<< FAILURE!
java.lang.AssertionError: webSocket.isClosed()
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.java_websocket.issues.Issue677Test.testIssue(Issue677Test.java:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:242)
at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:137)
at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Steps to reproduce the behavior:
I run the following command 10,000 times without modifying anything.
mvn test -Dtest=-Dtest=org.java_websocket.issues.Issue677Test#testIssue
This test is also possible to make consistently fail if we inject 100ms delay before the line 567 of the class WebSocketImpl.java
Expected behavior:
Test failure should not occur
Additional context:
There are many threads running with this test. Once the server instance starts its execution, one thread is responsible to set the value of ReadyState=Closed. However, due to some delay or bad thread interleaving, it slows the execution of that thread to set the value in the ReadyState. On the other hand, another thread that runs the test method finishes it's waiting time, and the assertion (webSocket0.isClosed()) is executed. As a result, the ReadyState.isClosed() returns false, and the assertion fails.
Environment(please complete the following information):
I ran the test on an Ubuntu 20.04 LTS machine using OpenJDK 1.8.0_312.
Describe the bug I ran this test 10,000 times in normal settings and got the following failures for many times.
Debug log Running org.java_websocket.issues.Issue677Test Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.125 sec <<< FAILURE! testIssue(org.java_websocket.issues.Issue677Test) Time elapsed: 0.103 sec <<< FAILURE! java.lang.AssertionError: webSocket.isClosed() at org.junit.Assert.fail(Assert.java:88) at org.junit.Assert.assertTrue(Assert.java:41) at org.java_websocket.issues.Issue677Test.testIssue(Issue677Test.java:119) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:242) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:137) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Results : Failed tests: testIssue(org.java_websocket.issues.Issue677Test): webSocket.isClosed() Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
Steps to reproduce the behavior: I run the following command 10,000 times without modifying anything. mvn test -Dtest=-Dtest=org.java_websocket.issues.Issue677Test#testIssue
This test is also possible to make consistently fail if we inject 100ms delay before the line 567 of the class WebSocketImpl.java
Expected behavior: Test failure should not occur
Additional context: There are many threads running with this test. Once the server instance starts its execution, one thread is responsible to set the value of ReadyState=Closed. However, due to some delay or bad thread interleaving, it slows the execution of that thread to set the value in the ReadyState. On the other hand, another thread that runs the test method finishes it's waiting time, and the assertion (webSocket0.isClosed()) is executed. As a result, the ReadyState.isClosed() returns false, and the assertion fails.
Environment(please complete the following information): I ran the test on an Ubuntu 20.04 LTS machine using OpenJDK 1.8.0_312.