Describe the bug

When I utilize Pulsar Java Admin API to get the topic's last message-id, I got 500 HTTP status code from Java API. As a comparison, I use pulsar-admin topics get-message-by-id ..., it seems OK.

To Reproduce

I cannot reproduce it, it'll happen occasionally after several hours running.

Behavior stated

Java API side

My getTopicLastMessage API snippet:

    static Optional<Message<byte[]>> getTopicLastMessage(
            PulsarAdmin admin,
            String fqTopic) throws PulsarAdminException
    {
        MessageIdImpl msgId = (MessageIdImpl) admin.topics().getLastMessageId(fqTopic);
        System.out.println("Topic last msg id: " + msgId);
        Message<byte[]> msg = admin.topics().getMessageById(fqTopic, msgId.getLedgerId(), msgId.getEntryId());
        return Optional.of(msg);
    }

When I call this API with topic persistent://foobar/mysql-cdc/binlog, I got the following logs:

My App logs...

``` 2021-07-29 10:28:02.612 INFO foobar.mysql.cdc.binlog.CdcClient: Getting topic last GTID from Pulsar ... Topic last msg id: -1:-1:-1 2021-07-29 10:28:03.415 ERROR foobar.mysql.cdc.binlog.CdcClient$EnsureThread$1: Cannot ensure binlog liveness foobar.mysql.cdc.exception.FatalException: Cannot get last message GTID from persistent://foobar/mysql-cdc/binlog at foobar.mysql.cdc.pulsar.PulsarAdminClient.getTopicLastGtidOrPanic(PulsarAdminClient.java:72) ~[classes/:?] at foobar.mysql.cdc.binlog.CdcClient.newBinlogClient(CdcClient.java:188) ~[classes/:?] at foobar.mysql.cdc.binlog.CdcClient.access$900(CdcClient.java:51) ~[classes/:?] at foobar.mysql.cdc.binlog.CdcClient$EnsureThread.ensure(CdcClient.java:364) ~[classes/:?] at foobar.mysql.cdc.binlog.CdcClient$EnsureThread.access$1000(CdcClient.java:342) ~[classes/:?] at foobar.mysql.cdc.binlog.CdcClient$EnsureThread$1.run(CdcClient.java:376) [classes/:?] at java.util.TimerThread.mainLoop(Timer.java:555) [?:1.8.0_292] at java.util.TimerThread.run(Timer.java:505) [?:1.8.0_292] Caused by: org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Errorjava.io.ByteArrayInputStream@46f2a7f5 at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:208) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:251) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl.getMessagesFromHttpResponse(TopicsImpl.java:1419) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl.access$100(TopicsImpl.java:90) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl$17.completed(TopicsImpl.java:1143) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl$17.completed(TopicsImpl.java:1139) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:814) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:212) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_292] at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$3(AsyncHttpConnector.java:253) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_292] at org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[pulsar-client-2.7.2.jar:2.7.2] at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292] Caused by: org.apache.pulsar.shade.javax.ws.rs.ServerErrorException: HTTP 500 Internal Server Error at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:244) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl.getMessagesFromHttpResponse(TopicsImpl.java:1419) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl.access$100(TopicsImpl.java:90) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl$17.completed(TopicsImpl.java:1143) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl$17.completed(TopicsImpl.java:1139) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:814) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:212) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_292] at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$3(AsyncHttpConnector.java:253) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_292] at org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[pulsar-client-2.7.2.jar:2.7.2] at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292] ```

Pulsar logs...

``` 02:28:01.603 [pulsar-web-63-24] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [29/Jul/2021:02:28:01 +0000] "GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false&subscriptionBacklogSize=false HTTP/1.1" 200 1950 "-" "Pulsar-Java-v2.8.0" 1 02:28:01.604 [pulsar-web-63-14] INFO org.eclipse.jetty.server.RequestLog - 127.0.0.1 - - [29/Jul/2021:02:28:01 +0000] "GET /admin/v2/persistent/public/functions/coordinate/stats?getPreciseBacklog=false&subscriptionBacklogSize=false HTTP/1.1" 200 1950 "-" "Pulsar-Java-v2.8.0" 1 02:28:03.347 [pulsar-web-63-24] INFO org.eclipse.jetty.server.RequestLog - 10.0.2.100 - - [29/Jul/2021:02:28:03 +0000] "GET /admin/v2/persistent/foobar/mysql-cdc/binlog/lastMessageId HTTP/1.1" 200 48 "-" "Pulsar-Java-v2.7.2" 1 02:28:03.409 [pulsar-web-63-14] ERROR org.apache.bookkeeper.client.impl.OpenBuilderBase - invalid ledgerId -1 < 0 02:28:03.409 [BookKeeperClientWorker-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [foobar/mysql-cdc/persistent/binlog] Error opening ledger for reading at position -1:-1 - org.apache.bookkeeper.mledger.ManagedLedgerException$NonRecoverableLedgerException: No such ledger exists on Metadata Server 02:28:03.410 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.eclipse.jetty.server.RequestLog - 10.0.2.100 - - [29/Jul/2021:02:28:03 +0000] "GET /admin/v2/persistent/foobar/mysql-cdc/binlog/ledger/-1/entry/-1 HTTP/1.1" 500 243 "-" "Pulsar-Java-v2.7.2" 5 ```

Note the HTTP/1.1" 500 error log.

Pulsar Admin command-line tool side

$ pulsar-admin topics last-message-id persistent://foobar/mysql-cdc/binlog
Warning: Nashorn engine is planned to be removed from a future JDK release
{
  "ledgerId" : -1,
  "entryId" : -1,
  "partitionIndex" : -1
}

02:52:16.047 [pulsar-web-63-14] INFO  org.eclipse.jetty.server.RequestLog - 10.0.2.100 - - [29/Jul/2021:02:52:16 +0000] "GET /admin/v2/persistent/foobar/mysql-cdc/binlog/lastMessageId HTTP/1.1" 200 48 "-" "Pulsar-Java-v2.7.2" 2

$ pulsar-admin topics get-message-by-id persistent://foobar/myqsl-cdc/binlog -e -1 -l -1
Warning: Nashorn engine is planned to be removed from a future JDK release
10:53:08.534 [AsyncHttpClient-7-1] WARN  org.apache.pulsar.client.admin.internal.TopicsImpl - Exception 'Policies not found for foobar/myqsl-cdc namespace' occurred while trying to get message.
Cannot find any messages based on ledgerId:-1 entryId:-1

02:53:08.486 [ForkJoinPool.commonPool-worker-5] WARN  org.apache.pulsar.broker.web.PulsarWebResource - Policies not found for foobar/myqsl-cdc namespace
02:53:08.486 [pulsar-web-63-14] ERROR org.apache.pulsar.broker.admin.impl.PersistentTopicsBase - [null] Failed to get message with ledgerId -1 entryId -1 from persistent://foobar/myqsl-cdc/binlog
org.apache.pulsar.broker.web.RestException: Policies not found for foobar/myqsl-cdc namespace
    at org.apache.pulsar.broker.web.PulsarWebResource.lambda$checkLocalOrGetPeerReplicationCluster$11(PulsarWebResource.java:760) ~[org.apache.pulsar-pulsar-broker-2.8.0.jar:2.8.0]
    at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714) ~[?:?]
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
    at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) ~[?:?]
    at org.apache.pulsar.zookeeper.ZooKeeperDataCache.lambda$getAsync$0(ZooKeeperDataCache.java:68) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:2.8.0]
    at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714) ~[?:?]
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
    at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) ~[?:?]
    at org.apache.pulsar.zookeeper.ZooKeeperCache.lambda$getDataAsync$17(ZooKeeperCache.java:388) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:2.8.0]
    at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714) ~[?:?]
    at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) ~[?:?]
    at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) ~[?:?]
    at org.apache.pulsar.zookeeper.ZooKeeperCache.lambda$getDataAsync$13(ZooKeeperCache.java:373) ~[org.apache.pulsar-pulsar-zookeeper-utils-2.8.0.jar:2.8.0]
    at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426) ~[?:?]
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) ~[?:?]
    at java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) ~[?:?]
    at java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) ~[?:?]
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) ~[?:?]
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) ~[?:?]
02:53:08.487 [pulsar-web-63-14] INFO  org.eclipse.jetty.server.RequestLog - 10.0.2.100 - - [29/Jul/2021:02:53:08 +0000] "GET /admin/v2/persistent/foobar/myqsl-cdc/binlog/ledger/-1/entry/-1 HTTP/1.1" 404 64 "-" "Pulsar-Java-v2.7.2" 2

What is weird to me is that last-message-id returned a ledgerId which is negative.

Desktop (please complete the following information):

OS: Ubuntu 20.04 LTS amd64
Maven artifacts: pulsar-client, pulsar-client-admin 2.7.2

Additional context

The Pulsar is running in Podman 3.2.3 with no special config

podman run \
    --name pulsar-cdc \
    --hostname pulsar-cdc \
    -d --restart=always \
    -p 6650:6650 -p 8080:8080 \
    apachepulsar/pulsar:latest \
    bin/pulsar standalone

$ podman image ls | grep pulsar
docker.io/apachepulsar/pulsar                         latest               0622425a7cd8  6 weeks ago   1.48 GB

$ podman logs pulsar-cdc 2> /dev/null | grep -i "version: "
02:42:58.484 [main] INFO  org.apache.pulsar.broker.PulsarService - Starting Pulsar Broker service; version: '2.8.0'

When I call getTopicLastMessage() again(after Pulsar container restarted) in my App:

App logs ...

``` 2021-07-29 11:38:45.210 INFO foobar.mysql.cdc.binlog.CdcClient: Getting topic last GTID from Pulsar ... Topic last msg id: 4716:-1:-1 2021-07-29 11:38:46.005 ERROR foobar.mysql.cdc.binlog.CdcClient$EnsureThread$1: Cannot ensure binlog liveness foobar.mysql.cdc.exception.FatalException: Cannot get last message GTID from persistent://foobar/mysql-cdc/binlog at foobar.mysql.cdc.pulsar.PulsarAdminClient.getTopicLastGtidOrPanic(PulsarAdminClient.java:72) ~[classes/:?] at foobar.mysql.cdc.binlog.CdcClient.newBinlogClient(CdcClient.java:188) ~[classes/:?] at foobar.mysql.cdc.binlog.CdcClient.access$900(CdcClient.java:51) ~[classes/:?] at foobar.mysql.cdc.binlog.CdcClient$EnsureThread.ensure(CdcClient.java:364) ~[classes/:?] at foobar.mysql.cdc.binlog.CdcClient$EnsureThread.access$1000(CdcClient.java:342) ~[classes/:?] at foobar.mysql.cdc.binlog.CdcClient$EnsureThread$1.run(CdcClient.java:376) [classes/:?] at java.util.TimerThread.mainLoop(Timer.java:555) [?:1.8.0_292] at java.util.TimerThread.run(Timer.java:505) [?:1.8.0_292] Caused by: org.apache.pulsar.client.admin.PulsarAdminException$ServerSideErrorException: HTTP 500 Internal Server Errorjava.io.ByteArrayInputStream@3bc1bbb at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:208) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:251) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl.getMessagesFromHttpResponse(TopicsImpl.java:1419) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl.access$100(TopicsImpl.java:90) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl$17.completed(TopicsImpl.java:1143) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl$17.completed(TopicsImpl.java:1139) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:814) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:212) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_292] at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$3(AsyncHttpConnector.java:253) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_292] at org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[pulsar-client-2.7.2.jar:2.7.2] at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292] Caused by: org.apache.pulsar.shade.javax.ws.rs.ServerErrorException: HTTP 500 Internal Server Error at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:244) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl.getMessagesFromHttpResponse(TopicsImpl.java:1419) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl.access$100(TopicsImpl.java:90) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl$17.completed(TopicsImpl.java:1143) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.TopicsImpl$17.completed(TopicsImpl.java:1139) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:814) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:212) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_292] at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$3(AsyncHttpConnector.java:253) ~[pulsar-client-admin-2.7.2.jar:2.7.2] at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_292] at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975) ~[?:1.8.0_292] at org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:324) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:296) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[pulsar-client-2.7.2.jar:2.7.2] at org.apache.pulsar.shade.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[pulsar-client-2.7.2.jar:2.7.2] at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292] ```

Which the error message is similar to the result of pulsar-admin topics get-message-by-id persistent://foobar/mysql-cdc/binlog -l 4716 -e -1 shown above.

apache / pulsar

Pulsar Java 2.7.2 admin API topics().getLastMessageId() return 500 HTTP status code #11491