I'm getting error logs in pulsar bookkeeper and because of this I'm not getting acknowledged from the pulsar whether the job is pushed successfully or not. Since there is no acknowledgment the jobs are lost once the instance is restarted. Following is the error trace,
Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.013 [pulsar-io-23-13] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37758 lastEntry:37758
Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.439 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37760 lastEntry:37760
Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.540 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37761 lastEntry:37761
Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.839 [pulsar-io-23-2] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762
Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.014 [pulsar-io-23-13] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762
Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.214 [pulsar-io-23-4] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42118 firstEntry:866 lastEntry:866
Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.240 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762
Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.414 [pulsar-io-23-13] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762
Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.440 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762
I have checked the same exception in the bookkeeper code,
From this it is understood that the lastEntry is greater than the lastAddConfirmed. We aren't sure why this is happening, is there a way we can over come it ?
Original Issue: apache/pulsar#13642
Hi,
I'm getting error logs in pulsar bookkeeper and because of this I'm not getting acknowledged from the pulsar whether the job is pushed successfully or not. Since there is no acknowledgment the jobs are lost once the instance is restarted. Following is the error trace,
Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.013 [pulsar-io-23-13] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37758 lastEntry:37758 Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.439 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37760 lastEntry:37760 Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.540 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37761 lastEntry:37761 Dec 21 11:45:33 ip-10-60-27-174 pulsar[2024114]: 11:45:33.839 [pulsar-io-23-2] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762 Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.014 [pulsar-io-23-13] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762 Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.214 [pulsar-io-23-4] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42118 firstEntry:866 lastEntry:866 Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.240 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762 Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.414 [pulsar-io-23-13] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762 Dec 21 11:45:34 ip-10-60-27-174 pulsar[2024114]: 11:45:34.440 [pulsar-io-23-5] ERROR org.apache.bookkeeper.client.LedgerHandle - ReadException on ledgerId:42132 firstEntry:37762 lastEntry:37762
I have checked the same exception in the bookkeeper code, From this it is understood that the lastEntry is greater than the lastAddConfirmed. We aren't sure why this is happening, is there a way we can over come it ?