[Bug] return topic partitions were not yet created when get last messageId

[graysonzeng]

Search before asking

• [X] I searched in the issues and found nothing similar.

Version

pulsar 3.1.1

Minimal reproduce step

1. The data in topic has expired
2. Rolling restart all brokers

What did you expect to see?

report the data has expired

What did you see instead?

Topic partitions were not yet created

14:51:29.217 [bookkeeper-ml-scheduler-OrderedScheduler-7-0] ERROR org.apache.pulsar.broker.admin.impl.PersistentTopicsBase - [admin] Failed to get last messageId persistent://pulsar/default/MusicActSweepStage-partition-3

ava.util.concurrent.CompletionException: org.apache.pulsar.broker.web.RestException: Topic partitions were not yet created
 java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) ~[?:?]
 java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:687) ~[?:?]
 java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:662) ~[?:?]
 java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:2168) ~[?:?]
 org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.topicNotFoundReasonAsync(PersistentTopicsBase.java:4419) ~[pulsar-broker.jar:3.1.1]
 org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.lambda$getTopicReferenceAsync$422(PersistentTopicsBase.java:4399) ~[pulsar-broker.jar:3.1.1]
 java.util.Optional.orElseGet(Optional.java:364) ~[?:?]
 org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.lambda$getTopicReferenceAsync$423(PersistentTopicsBase.java:4399) ~[pulsar-broker.jar:3.1.1]
 java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1150) ~[?:?]
 java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
 java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147) ~[?:?]
 org.apache.pulsar.broker.service.BrokerService$2.openLedgerFailed(BrokerService.java:1800) ~[pulsar-broker.jar:3.1.1]
 org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl.lambda$asyncOpen$8(ManagedLedgerFactoryImpl.java:428) ~[org.apache.pulsar-managed-ledger-3.1.1.jar:3.1.1]
 java.util.concurrent.CompletableFuture.uniExceptionally(CompletableFuture.java:990) ~[?:?]
 java.util.concurrent.CompletableFuture$UniExceptionally.tryFire(CompletableFuture.java:974) ~[?:?]
 java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
 java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) ~[?:?]
 org.apache.bookkeeper.mledger.impl.ManagedLedgerFactoryImpl$2.initializeFailed(ManagedLedgerFactoryImpl.java:423) ~[org.apache.pulsar-managed-ledger-3.1.1.jar:3.1.1]
 org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl$1.operationFailed(ManagedLedgerImpl.java:457) ~[org.apache.pulsar-managed-ledger-3.1.1.jar:3.1.1]
 org.apache.bookkeeper.mledger.impl.MetaStoreImpl.lambda$getManagedLedgerInfo$3(MetaStoreImpl.java:135) ~[org.apache.pulsar-managed-ledger-3.1.1.jar:3.1.1]
 java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:718) ~[?:?]
 java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482) ~[?:?]
 org.apache.bookkeeper.common.util.OrderedExecutor$TimedRunnable.run(OrderedExecutor.java:201) ~[org.apache.bookkeeper-bookkeeper-common-4.16.3.jar:4.16.3]
 org.apache.bookkeeper.common.util.SingleThreadSafeScheduledExecutorService$SafeRunnable.run(SingleThreadSafeScheduledExecutorService.java:46) ~[org.apache.bookkeeper-bookkeeper-common-4.16.3.jar:4.16.3]
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
 java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
 java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
 io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-common-4.1.100.Final.jar:4.1.100.Final]
 java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: org.apache.pulsar.broker.web.RestException: Topic partitions were not yet created
 org.apache.pulsar.broker.admin.impl.PersistentTopicsBase.lambda$topicNotFoundReasonAsync$426(PersistentTopicsBase.java:4421) ~[pulsar-broker.jar:3.1.1]
 java.util.concurrent.CompletableFuture.uniApplyNow(CompletableFuture.java:684) ~[?:?]
... 29 more

Anything else?

No response

Are you willing to submit a PR?

• [ ] I'm willing to submit a PR!

[Technoboy-]

MusicActSweepStage-partition-3 is new added after topic created?

[graysonzeng]

MusicActSweepStage-partition-3 is new added after topic created?

No, the topic was created a long time ago，and we didn't add a new partition

[Technoboy-]

does the topic has active consumers/producers?

[graysonzeng]

does the topic has active consumers/producers?

I observed that there are 3 active readers and no producers

[Technoboy-]

does the topic has active consumers/producers?

I observed that there are 3 active readers and no producers

you can check the broker log why this partition be deleted.

[Technoboy-]

it maybe related to the retention policies.

[graysonzeng]

it maybe related to the retention policies.

This idea makes sense, but I think even if the message is expired, it should return a correct message, for example, the message may be expired, or id=-1, instead of Topic partitions were not yet created