apache / pulsar

Apache Pulsar - distributed pub-sub messaging system
https://pulsar.apache.org/
Apache License 2.0
14.23k stars 3.58k forks source link

Flaky-test: NamespaceAuthZTest.testPermission #22466

Open lhotari opened 6 months ago

lhotari commented 6 months ago

Search before asking

Example failure

https://github.com/apache/pulsar/actions/runs/8614618958/job/23608820624?pr=22464#step:11:1452

Exception stacktrace

  Error:  Tests run: 31, Failures: 1, Errors: 0, Skipped: 24, Time elapsed: 14.447 s <<< FAILURE! - in org.apache.pulsar.broker.admin.NamespaceAuthZTest
  Error:  org.apache.pulsar.broker.admin.NamespaceAuthZTest.testPermission  Time elapsed: 0.18 s  <<< FAILURE!
  org.mockito.exceptions.misusing.UnfinishedStubbingException: 

  Unfinished stubbing detected here:
  -> at org.apache.pulsar.broker.admin.NamespaceAuthZTest.setAuthorizationOperationChecker(NamespaceAuthZTest.java:135)

  E.g. thenReturn() may be missing.
  Examples of correct stubbing:
      when(mock.isOk()).thenReturn(true);
      when(mock.isOk()).thenThrow(exception);
      doThrow(exception).when(mock).someVoidMethod();
  Hints:
   1. missing thenReturn()
   2. you are trying to stub a final method, which is not supported
   3. you are stubbing the behaviour of another mock inside before 'thenReturn' instruction is completed

    at org.apache.pulsar.broker.authorization.AuthorizationService.allowNamespaceOperationAsync(AuthorizationService.java:572)
    at org.apache.pulsar.broker.admin.NamespaceAuthZTest.setAuthorizationOperationChecker(NamespaceAuthZTest.java:142)
    at org.apache.pulsar.broker.admin.NamespaceAuthZTest.testPermission(NamespaceAuthZTest.java:582)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
    at org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
    at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
    at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)

Mockito isn't thread safe and this could be a problem caused by invalid use of Mockito in multi-threaded code.

Are you willing to submit a PR?

lhotari commented 6 months ago

This might be related to #22398 changes.

lhotari commented 6 months ago

Recent issue:

  Error:  Tests run: 31, Failures: 1, Errors: 0, Skipped: 24, Time elapsed: 16.162 s <<< FAILURE! - in org.apache.pulsar.broker.admin.NamespaceAuthZTest
  Error:  org.apache.pulsar.broker.admin.NamespaceAuthZTest.testPermission  Time elapsed: 0.185 s  <<< FAILURE!
  org.mockito.exceptions.misusing.UnfinishedStubbingException: 

  Unfinished stubbing detected here:
  -> at org.apache.pulsar.broker.admin.NamespaceAuthZTest.setAuthorizationOperationChecker(NamespaceAuthZTest.java:135)

  E.g. thenReturn() may be missing.
  Examples of correct stubbing:
      when(mock.isOk()).thenReturn(true);
      when(mock.isOk()).thenThrow(exception);
      doThrow(exception).when(mock).someVoidMethod();
  Hints:
   1. missing thenReturn()
   2. you are trying to stub a final method, which is not supported
   3. you are stubbing the behaviour of another mock inside before 'thenReturn' instruction is completed

    at org.apache.pulsar.broker.authorization.AuthorizationService.allowNamespaceOperationAsync(AuthorizationService.java:572)
    at org.apache.pulsar.broker.admin.NamespaceAuthZTest.setAuthorizationOperationChecker(NamespaceAuthZTest.java:142)
    at org.apache.pulsar.broker.admin.NamespaceAuthZTest.testPermission(NamespaceAuthZTest.java:582)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
    at org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
    at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
    at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)

in https://github.com/apache/pulsar/actions/runs/8736977834/job/23977794669?pr=22532#step:11:1573

lhotari commented 6 months ago

This is a similar Mockito multi-threading issue as #22422 which was addressed by #22507. @Technoboy- or @coderzc do you have a chance to fix this?

lhotari commented 5 months ago

Again in https://github.com/apache/pulsar/actions/runs/9337942538/job/25700560029?pr=22818#step:11:1507

  Error:  Tests run: 54, Failures: 1, Errors: 0, Skipped: 7, Time elapsed: 40.781 s <<< FAILURE! - in org.apache.pulsar.broker.admin.NamespaceAuthZTest
  Error:  org.apache.pulsar.broker.admin.NamespaceAuthZTest.testSubscriptionExpirationTime  Time elapsed: 0.025 s  <<< FAILURE!
  org.mockito.exceptions.misusing.UnfinishedStubbingException: 

  Unfinished stubbing detected here:
  -> at org.apache.pulsar.broker.admin.NamespaceAuthZTest.setAuthorizationPolicyOperationChecker(NamespaceAuthZTest.java:173)

  E.g. thenReturn() may be missing.
  Examples of correct stubbing:
      when(mock.isOk()).thenReturn(true);
      when(mock.isOk()).thenThrow(exception);
      doThrow(exception).when(mock).someVoidMethod();
  Hints:
   1. missing thenReturn()
   2. you are trying to stub a final method, which is not supported
   3. you are stubbing the behaviour of another mock inside before 'thenReturn' instruction is completed

    at org.apache.pulsar.broker.authorization.AuthorizationService.allowNamespacePolicyOperationAsync(AuthorizationService.java:615)
    at org.apache.pulsar.broker.admin.NamespaceAuthZTest.setAuthorizationPolicyOperationChecker(NamespaceAuthZTest.java:183)
    at org.apache.pulsar.broker.admin.NamespaceAuthZTest.testSubscriptionExpirationTime(NamespaceAuthZTest.java:1807)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
    at org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
    at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
    at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)