search

milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
31.02k stars 2.95k forks source link

[Bug]: querynode cpu larger and larger #34270

Closed lzhin closed 3 months ago

lzhin commented 5 months ago

Is there an existing issue for this?

Environment

- Milvus version:2.4.5
- Deployment mode(standalone or cluster):cluster
- MQ type(rocksmq, pulsar or kafka):    pulsar 
- SDK version(e.g. pymilvus v2.0.0rc2):pymilvus v2.3.7
- OS(Ubuntu or CentOS): Ubuntu 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

as the emb insert to collection , the querynode cpu larger and larger. it seems some index not build in time image

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

yanliang567 commented 5 months ago

generally speaking, insertion does not matter to the cpu usage of quernode, it could impact the cpu usage of datanode. so quick questions:

  1. how did you do inserting? batch size for each request, num of insert concurrencies?
  2. did you also do seaching at the same time?
  3. please offer all the milvus pods logs, and more sceenshots of milvus metrics /assign @lzhin /unassign
lzhin commented 5 months ago

generally speaking, insertion does not matter to the cpu usage of quernode, it could impact the cpu usage of datanode. so quick questions:

  1. how did you do inserting? batch size for each request, num of insert concurrencies?
  2. did you also do seaching at the same time?
  3. please offer all the milvus pods logs, and more sceenshots of milvus metrics /assign @lzhin /unassign

1.insert batch , batch is 119 ,emb size is 128. and the concurrencies may be 1500 2.yes , also search while insert 3.I did not deploy by k8s or docker compose. so hard to export log.

lzhin commented 5 months ago

image it seem some segment not seed in time ,so the search is by violence？

yanliang567 commented 5 months ago
  1. If a few segments are not sealed or indexed in time, Milvus has temp-index to avoid brute force similarity searching.
  2. if you have searching, the cpu usage of querynode is expected.
lzhin commented 5 months ago

2. expected

  1. If a few segments are not sealed or indexed in time, Milvus has temp-index to avoid brute force similarity searching.
  2. if you have searching, the cpu usage of querynode is expected.

but the cpu is larger and larger ,even up to 100%. and many search is timeout. I try to release the collection and rebuild the index. and then the search is normal, the cpu is not high.

xiaofan-luan commented 5 months ago

you might need to increase index node number if there are many segment lefted unindexed

xiaofan-luan commented 5 months ago

or use ivf since it's build speed is much faster compared to hnsw

lzhin commented 5 months ago

I think the indexnoed is enough , original the indexnode num is 5. and sometime the cpu is 0. the indexnode is 16 core 16g. I try use 12 indexnodes . but the querynode cpu is also high. it might not nothing to do with indexnode.

lzhin commented 5 months ago

I use IVF_SQ8 index type

yanliang567 commented 5 months ago

what about the memory usage of quernodes at that moment? We still need full milvus logs to know what was happening, or we could just guess everything...:(

lzhin commented 5 months ago

the memory is normal.
image image to avoid cpu 100% , I can only rebuild the partition every 2 hours.

lzhin commented 5 months ago

whether have other method to export the log, I can enter every node to run some command.

lzhin commented 5 months ago

the print log is: querynode: I20240629 17:18:22.499850 90135 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450768056431103236 I20240629 17:18:22.499919 90135 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103236, fieldID:0 I20240629 17:18:22.499822 90077 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450768056431103236 I20240629 17:18:22.499822 90079 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 102 of segment 450768056431103236 I20240629 17:18:22.499884 90106 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103236, fieldID:1 I20240629 17:18:22.499820 90110 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450768056431103236 I20240629 17:18:22.499989 90079 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103236, fieldID:102 I20240629 17:18:22.499963 90077 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103236, fieldID:101 I20240629 17:18:22.500025 90110 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103236, fieldID:100 I20240629 17:18:22.499914 90144 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103236, fieldID:104 I20240629 17:18:22.499855 90104 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 103 of segment 450768056431103236 I20240629 17:18:22.500102 90104 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103236, fieldID:103 I20240629 17:18:22.588011 90144 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103236, fieldID:104 I20240629 17:18:22.590881 90034 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103238, fieldID:0 I20240629 17:18:22.606957 90106 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103236, fieldID:1 I20240629 17:18:22.622954 90101 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103238, fieldID:101 I20240629 17:18:22.627303 90126 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103238, fieldID:104 I20240629 17:18:22.628551 90126 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450768056431103237 I20240629 17:18:22.628712 90126 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103237, fieldID:101 I20240629 17:18:22.628708 90101 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450768056431103237 I20240629 17:18:22.628768 90034 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 102 of segment 450768056431103237 I20240629 17:18:22.628914 90034 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103237, fieldID:102 I20240629 17:18:22.628823 90144 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450768056431103237 I20240629 17:18:22.628854 90101 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103237, fieldID:100 I20240629 17:18:22.628752 90106 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450768056431103237 I20240629 17:18:22.628974 90144 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103237, fieldID:0 I20240629 17:18:22.629092 90106 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056431103237, fieldID:1 I20240629 17:18:22.637818 90070 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103238, fieldID:100 I20240629 17:18:22.654150 90110 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103236, fieldID:100 I20240629 17:18:22.723970 90011 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103238, fieldID:1 I20240629 17:18:22.732656 90077 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103236, fieldID:101 I20240629 17:18:22.751984 90144 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103237, fieldID:0 I20240629 17:18:22.810554 90106 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103237, fieldID:1 I20240629 17:18:22.926837 90135 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103236, fieldID:0 I20240629 17:18:22.932545 90126 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103237, fieldID:101 I20240629 17:18:22.968084 90104 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103236, fieldID:103 I20240629 17:18:22.976140 90079 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:18:22.976208 90079 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:18:23.208745 90116 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103238, fieldID:103 I20240629 17:18:23.431219 90090 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:18:23.431283 90090 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:18:23.514438 90101 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103237, fieldID:100 I20240629 17:18:23.518287 90034 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:18:23.518352 90034 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:18:25.039456 90079 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (2063.161841 ms) I20240629 17:18:25.043283 90079 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103236, fieldID:102 I20240629 17:18:25.299436 90090 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1868.079483 ms) I20240629 17:18:25.302875 90090 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103238, fieldID:102 I20240629 17:18:25.495656 90034 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1977.228846 ms) I20240629 17:18:25.499428 90034 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056431103237, fieldID:102 I20240629 17:18:32.518682 90091 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:18:32.518738 90091 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 7 I20240629 17:18:32.518754 90091 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:18:32.811749 90091 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:18:32.811805 90091 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:18:32.811818 90091 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:18:32.858882 90091 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (47.005948 ms) I20240629 17:18:32.858948 90091 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:18:33.146355 90205 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450767622823184815 I20240629 17:18:33.146358 90111 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450767622823184815 I20240629 17:18:33.146365 90219 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450767622823184815 I20240629 17:18:33.146358 90212 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450767622823184815 I20240629 17:18:33.146754 90111 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823184815, fieldID:101 I20240629 17:18:33.146778 90212 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823184815, fieldID:0 I20240629 17:18:33.146456 90205 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823184815, fieldID:1 I20240629 17:18:33.146785 90219 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823184815, fieldID:100 I20240629 17:18:33.247146 90205 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823184815, fieldID:1 I20240629 17:18:33.253101 90111 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823184815, fieldID:101 I20240629 17:18:33.257607 90212 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823184815, fieldID:0 I20240629 17:18:33.369611 90219 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823184815, fieldID:100 I20240629 17:18:34.756367 90224 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1186.591534 ms) I20240629 17:18:34.774088 90098 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1204.908185 ms) I20240629 17:18:41.984179 90230 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:18:41.984246 90230 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 3 I20240629 17:18:41.984264 90230 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:18:41.986479 90239 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:18:41.986538 90239 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 3 I20240629 17:18:41.986555 90239 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:18:42.102643 90230 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:18:42.102705 90230 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:18:42.102716 90230 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:18:42.108759 90239 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:18:42.108816 90239 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:18:42.108827 90239 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:18:42.110431 90230 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (7.612520 ms) I20240629 17:18:42.110504 90230 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:18:42.118894 90239 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (9.998488 ms) I20240629 17:18:42.118976 90239 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:18:51.981941 90218 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:18:51.981995 90218 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 3 I20240629 17:18:51.982010 90218 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:18:52.099346 90218 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:18:52.099404 90218 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:18:52.099416 90218 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:18:52.108707 90218 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (9.225848 ms) I20240629 17:18:52.108773 90218 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:19:22.493549 90150 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:19:22.493604 90150 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 5 I20240629 17:19:22.493620 90150 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:19:22.644697 90150 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:19:22.644757 90150 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:19:22.644767 90150 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:19:22.670969 90150 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (26.137587 ms) I20240629 17:19:22.671044 90150 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:19:22.821583 90150 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450767622823470196 I20240629 17:19:22.821588 90203 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450767622823470196 I20240629 17:19:22.821583 90247 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450767622823470196 I20240629 17:19:22.821658 90150 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470196, fieldID:1 I20240629 17:19:22.821681 90203 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470196, fieldID:100 I20240629 17:19:22.821692 90247 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470196, fieldID:0 I20240629 17:19:22.821651 90214 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450767622823470196 I20240629 17:19:22.821771 90214 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470196, fieldID:101 I20240629 17:19:22.833643 90247 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470196, fieldID:0 I20240629 17:19:22.834265 90150 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470196, fieldID:1 I20240629 17:19:22.851447 90203 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470196, fieldID:100 I20240629 17:19:22.865244 90214 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470196, fieldID:101 I20240629 17:19:52.494483 90207 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:19:52.494549 90207 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 5 I20240629 17:19:52.494568 90207 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:19:52.642311 90207 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:19:52.642366 90207 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:19:52.642380 90207 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:19:52.668462 90207 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (25.994647 ms) I20240629 17:19:52.668534 90207 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:19:52.820767 90268 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450767622823027584 I20240629 17:19:52.820771 90207 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450767622823027584 I20240629 17:19:52.820771 90282 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450767622823027584 I20240629 17:19:52.820855 90268 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027584, fieldID:1 I20240629 17:19:52.820887 90207 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027584, fieldID:101 I20240629 17:19:52.820768 90141 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450767622823027584 I20240629 17:19:52.820925 90282 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027584, fieldID:0 I20240629 17:19:52.820963 90141 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027584, fieldID:100 I20240629 17:19:52.846998 90268 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027584, fieldID:1 I20240629 17:19:52.901051 90141 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027584, fieldID:100 I20240629 17:19:52.911289 90282 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027584, fieldID:0 I20240629 17:19:52.922185 90207 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027584, fieldID:101 I20240629 17:21:10.552877 90300 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:21:10.552942 90300 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:21:10.754247 90344 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:21:10.754307 90344 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:21:52.701646 90266 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:21:52.701711 90266 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 6 I20240629 17:21:52.701731 90266 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:21:52.911355 90266 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:21:52.911409 90266 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:21:52.911420 90266 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:21:52.946728 90266 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (35.243758 ms) I20240629 17:21:52.946812 90266 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:21:52.950881 90314 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450767622823027693 I20240629 17:21:52.950881 90297 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450767622823027693 I20240629 17:21:52.950881 90244 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450767622823027693 I20240629 17:21:52.950881 90334 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450767622823027693 I20240629 17:21:52.950958 90314 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027693, fieldID:1 I20240629 17:21:52.950973 90297 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027693, fieldID:0 I20240629 17:21:52.950989 90334 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027693, fieldID:100 I20240629 17:21:52.950973 90244 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027693, fieldID:101 I20240629 17:21:53.026780 90244 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027693, fieldID:101 I20240629 17:21:53.039233 90314 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027693, fieldID:1 I20240629 17:21:53.086711 90334 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027693, fieldID:100 I20240629 17:21:53.091598 90297 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027693, fieldID:0 I20240629 17:22:22.499363 90369 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:22:22.499419 90369 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 6 I20240629 17:22:22.499434 90369 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:22:22.709919 90369 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:22:22.709975 90369 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:22:22.709992 90369 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:22:22.745672 90369 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (35.607155 ms) I20240629 17:22:22.745760 90369 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:22:22.963742 90331 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450767622823470399 I20240629 17:22:22.963742 90369 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450767622823470399 I20240629 17:22:22.963999 90331 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470399, fieldID:0 I20240629 17:22:22.964004 90369 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470399, fieldID:101 I20240629 17:22:22.963948 90413 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450767622823470399 I20240629 17:22:22.964059 90413 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470399, fieldID:100 I20240629 17:22:22.964032 90412 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450767622823470399 I20240629 17:22:22.964114 90412 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470399, fieldID:1 I20240629 17:22:23.046531 90412 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470399, fieldID:1 I20240629 17:22:23.050138 90369 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470399, fieldID:101 I20240629 17:22:23.105525 90413 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470399, fieldID:100 I20240629 17:22:23.135797 90331 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470399, fieldID:0 I20240629 17:22:55.912685 90377 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1148.497929 ms) I20240629 17:22:58.172149 90377 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1208.547460 ms) I20240629 17:23:29.359200 90351 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:23:29.359283 90351 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:23:29.753520 90440 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:23:29.753584 90440 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:24:40.752142 90404 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:24:40.752215 90404 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:24:40.752297 90456 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:24:40.752379 90456 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:24:42.516323 90318 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450768056432706924 I20240629 17:24:42.516384 90318 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706924, fieldID:0 I20240629 17:24:42.516326 90449 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 102 of segment 450768056432706924 I20240629 17:24:42.516422 90449 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706924, fieldID:102 I20240629 17:24:42.516350 90237 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450768056432706924 I20240629 17:24:42.516343 90488 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450768056432706924 I20240629 17:24:42.516459 90237 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706924, fieldID:100 I20240629 17:24:42.516323 90503 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450768056432706924 I20240629 17:24:42.516466 90488 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706924, fieldID:101 I20240629 17:24:42.516516 90503 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706924, fieldID:1 I20240629 17:24:42.534554 90318 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706924, fieldID:0 I20240629 17:24:42.535769 90237 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706924, fieldID:100 I20240629 17:24:42.538095 90503 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706924, fieldID:1 I20240629 17:24:42.543167 90488 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706924, fieldID:101 I20240629 17:24:42.571071 90449 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:24:42.571136 90449 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:24:42.761065 90498 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:24:42.761130 90498 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 6 I20240629 17:24:42.761145 90498 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:24:42.968916 90498 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:24:42.968974 90498 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:24:42.968988 90498 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:24:43.010906 90498 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (41.853795 ms) I20240629 17:24:43.010982 90498 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:24:43.014906 90488 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450767622823470398 I20240629 17:24:43.014916 90318 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450767622823470398 I20240629 17:24:43.014933 90503 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450767622823470398 I20240629 17:24:43.014927 90237 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450767622823470398 I20240629 17:24:43.015022 90318 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470398, fieldID:0 I20240629 17:24:43.015082 90237 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470398, fieldID:101 I20240629 17:24:43.014986 90488 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470398, fieldID:1 I20240629 17:24:43.015048 90503 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470398, fieldID:100 I20240629 17:24:43.085038 90488 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470398, fieldID:1 I20240629 17:24:43.129642 90237 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470398, fieldID:101 I20240629 17:24:43.154807 90503 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470398, fieldID:100 I20240629 17:24:43.165263 90318 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470398, fieldID:0 I20240629 17:24:43.707406 90449 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1136.198537 ms) I20240629 17:24:43.708788 90449 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706924, fieldID:102 I20240629 17:24:51.575709 90532 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:24:51.575773 90532 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:24:51.981797 90492 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:24:51.981932 90492 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 1 I20240629 17:24:51.981995 90492 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 0 I20240629 17:24:52.025568 90492 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:24:52.025624 90492 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:24:52.025637 90492 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:24:52.029251 90492 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (3.529708 ms) I20240629 17:24:52.029289 90492 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:24:52.499665 90472 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:24:52.499712 90472 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 1 I20240629 17:24:52.499725 90472 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 0 I20240629 17:24:52.502194 90536 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:24:52.502252 90536 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 1 I20240629 17:24:52.502267 90536 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 0 I20240629 17:24:52.548465 90472 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:24:52.548532 90472 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:24:52.548545 90472 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:24:52.551795 90536 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:24:52.551853 90536 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:24:52.551868 90536 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:24:52.552397 90472 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (3.799358 ms) I20240629 17:24:52.552443 90472 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:24:52.555750 90536 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (3.829945 ms) I20240629 17:24:52.555789 90536 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:24:52.576118 90484 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450768056432706926 I20240629 17:24:52.576128 90492 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450768056432706926 I20240629 17:24:52.576184 90484 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706926, fieldID:1 I20240629 17:24:52.576159 90539 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450768056432706926 I20240629 17:24:52.576222 90492 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706926, fieldID:101 I20240629 17:24:52.576256 90539 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706926, fieldID:100 I20240629 17:24:52.576121 90504 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450768056432706926 I20240629 17:24:52.576310 90504 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706926, fieldID:0 I20240629 17:24:52.577431 90540 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450768056432706927 I20240629 17:24:52.577458 90472 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450768056432706927 I20240629 17:24:52.577471 90540 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706927, fieldID:1 I20240629 17:24:52.577453 90373 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450768056432706927 I20240629 17:24:52.577517 90472 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706927, fieldID:0 I20240629 17:24:52.577463 90536 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450768056432706927 I20240629 17:24:52.577549 90373 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706927, fieldID:101 I20240629 17:24:52.577566 90536 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432706927, fieldID:100 I20240629 17:24:52.593423 90484 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706926, fieldID:1 I20240629 17:24:52.593425 90504 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706926, fieldID:0 I20240629 17:24:52.595561 90540 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706927, fieldID:1 I20240629 17:24:52.596701 90539 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706926, fieldID:100 I20240629 17:24:52.597434 90472 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706927, fieldID:0 I20240629 17:24:52.598855 90492 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706926, fieldID:101 I20240629 17:24:52.599296 90536 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706927, fieldID:100 I20240629 17:24:52.604348 90373 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432706927, fieldID:101 I20240629 17:25:02.559202 90541 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450768056430500758 I20240629 17:25:02.559211 90558 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450768056430500758 I20240629 17:25:02.559278 90541 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430500758, fieldID:100 I20240629 17:25:02.559211 90506 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450768056430500758 I20240629 17:25:02.559307 90558 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430500758, fieldID:0 I20240629 17:25:02.559214 90567 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 102 of segment 450768056430500758 I20240629 17:25:02.559330 90506 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430500758, fieldID:1 I20240629 17:25:02.559268 90544 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450768056430500758 I20240629 17:25:02.559384 90567 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430500758, fieldID:102 I20240629 17:25:02.559418 90544 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430500758, fieldID:101 I20240629 17:25:02.564471 90534 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 102 of segment 450768056430500776 I20240629 17:25:02.564551 90534 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430500776, fieldID:102 I20240629 17:25:02.564513 90565 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450768056430500776 I20240629 17:25:02.564450 90547 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450768056430500776 I20240629 17:25:02.564465 90505 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450768056430500776 I20240629 17:25:02.564620 90565 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430500776, fieldID:0 I20240629 17:25:02.564457 90418 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450768056430500776 I20240629 17:25:02.564666 90547 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430500776, fieldID:100 I20240629 17:25:02.564742 90418 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430500776, fieldID:101 I20240629 17:25:02.564674 90505 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430500776, fieldID:1 I20240629 17:25:02.729826 90506 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430500758, fieldID:1 I20240629 17:25:02.743929 90505 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430500776, fieldID:1 I20240629 17:25:02.782371 90544 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430500758, fieldID:101 I20240629 17:25:02.806749 90541 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430500758, fieldID:100 I20240629 17:25:02.808084 90418 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430500776, fieldID:101 I20240629 17:25:02.834120 90565 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430500776, fieldID:0 I20240629 17:25:02.855849 90558 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430500758, fieldID:0 I20240629 17:25:02.910557 90547 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430500776, fieldID:100 I20240629 17:25:03.111843 90534 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:25:03.111917 90534 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:25:03.371169 90567 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:25:03.371237 90567 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:25:05.188191 90534 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (2076.191004 ms) I20240629 17:25:05.191577 90534 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430500776, fieldID:102 I20240629 17:25:05.428997 90567 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (2057.689223 ms) I20240629 17:25:05.432544 90567 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430500758, fieldID:102 I20240629 17:25:11.992244 90535 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:25:11.992300 90535 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 3 I20240629 17:25:11.992312 90535 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:25:11.993988 90543 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:25:11.994042 90543 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 3 I20240629 17:25:11.994058 90543 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:25:12.148105 90535 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:25:12.148159 90535 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:25:12.148175 90535 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:25:12.156838 90535 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (8.610626 ms) I20240629 17:25:12.156896 90535 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:25:12.161409 90543 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:25:12.161532 90543 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:25:12.161550 90543 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:25:12.175316 90543 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (13.661809 ms) I20240629 17:25:12.175411 90543 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:25:12.675218 90535 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450768056430700950 I20240629 17:25:12.675236 90580 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450768056430700950 I20240629 17:25:12.675218 90563 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450768056430700950 I20240629 17:25:12.675343 90580 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430700950, fieldID:101 I20240629 17:25:12.675346 90563 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430700950, fieldID:1 I20240629 17:25:12.675305 90535 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430700950, fieldID:0 I20240629 17:25:12.675235 90532 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 102 of segment 450768056430700950 I20240629 17:25:12.675226 90557 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450768056430700950 I20240629 17:25:12.675420 90532 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430700950, fieldID:102 I20240629 17:25:12.675424 90557 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056430700950, fieldID:100 I20240629 17:25:12.809669 90563 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430700950, fieldID:1 I20240629 17:25:12.851321 90580 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430700950, fieldID:101 I20240629 17:25:12.867936 90535 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430700950, fieldID:0 I20240629 17:25:12.910290 90557 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430700950, fieldID:100 I20240629 17:25:13.301115 90532 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:25:13.301191 90532 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:25:15.336382 90532 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (2035.120829 ms) I20240629 17:25:15.340371 90532 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056430700950, fieldID:102 I20240629 17:25:16.333122 90538 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1457.745086 ms) I20240629 17:25:16.375370 90502 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1497.748586 ms) I20240629 17:25:31.983989 90483 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:25:31.984054 90483 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 3 I20240629 17:25:31.984078 90483 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:25:32.108335 90483 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:25:32.108393 90483 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:25:32.108409 90483 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:25:32.117314 90483 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (8.852984 ms) I20240629 17:25:32.117375 90483 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:25:52.610884 90498 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:25:52.610940 90498 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 7 I20240629 17:25:52.610956 90498 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:25:52.878315 90498 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:25:52.878383 90498 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:25:52.878401 90498 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:25:52.922669 90498 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (44.175191 ms) I20240629 17:25:52.922741 90498 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:25:53.181309 90498 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450767622823470582 I20240629 17:25:53.181355 90543 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450767622823470582 I20240629 17:25:53.181385 90498 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470582, fieldID:0 I20240629 17:25:53.181420 90543 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470582, fieldID:100 I20240629 17:25:53.181325 90486 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450767622823470582 I20240629 17:25:53.181326 90585 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450767622823470582 I20240629 17:25:53.181470 90486 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470582, fieldID:1 I20240629 17:25:53.181496 90585 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823470582, fieldID:101 I20240629 17:25:53.257038 90486 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470582, fieldID:1 I20240629 17:25:53.301129 90585 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470582, fieldID:101 I20240629 17:25:53.363492 90498 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470582, fieldID:0 I20240629 17:25:53.404019 90543 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823470582, fieldID:100 I20240629 17:26:07.258584 90629 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:26:07.258644 90629 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:26:08.069440 90629 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:26:08.069500 90629 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:26:12.499269 90581 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450768056432707632 I20240629 17:26:12.499329 90537 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 103 of segment 450768056432707632 I20240629 17:26:12.499341 90634 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 102 of segment 450768056432707632 I20240629 17:26:12.499348 90626 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 104 of segment 450768056432707632 I20240629 17:26:12.499349 90621 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450768056432707632 I20240629 17:26:12.499334 90591 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450768056432707632 I20240629 17:26:12.499348 90581 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707632, fieldID:0 I20240629 17:26:12.499424 90634 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707632, fieldID:102 I20240629 17:26:12.499490 90591 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707632, fieldID:100 I20240629 17:26:12.499361 90601 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450768056432707632 I20240629 17:26:12.499395 90537 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707632, fieldID:103 I20240629 17:26:12.499482 90621 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707632, fieldID:101 I20240629 17:26:12.499433 90626 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707632, fieldID:104 I20240629 17:26:12.499570 90601 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707632, fieldID:1 I20240629 17:26:12.500780 90622 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450768056432707634 I20240629 17:26:12.500800 90616 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 103 of segment 450768056432707634 I20240629 17:26:12.500810 90592 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450768056432707634 I20240629 17:26:12.500809 90622 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707634, fieldID:100 I20240629 17:26:12.500833 90633 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 102 of segment 450768056432707634 I20240629 17:26:12.500810 90452 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450768056432707634 I20240629 17:26:12.500861 90448 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 104 of segment 450768056432707634 I20240629 17:26:12.500849 90616 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707634, fieldID:103 I20240629 17:26:12.500859 90592 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707634, fieldID:0 I20240629 17:26:12.500833 90597 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450768056432707634 I20240629 17:26:12.501366 90633 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707634, fieldID:102 I20240629 17:26:12.501370 90448 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707634, fieldID:104 I20240629 17:26:12.501395 90452 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707634, fieldID:101 I20240629 17:26:12.501413 90597 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707634, fieldID:1 I20240629 17:26:12.502223 90545 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450768056432707633 I20240629 17:26:12.502225 90404 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 103 of segment 450768056432707633 I20240629 17:26:12.502223 90586 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 104 of segment 450768056432707633 I20240629 17:26:12.502281 90631 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 102 of segment 450768056432707633 I20240629 17:26:12.502260 90629 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450768056432707633 I20240629 17:26:12.502259 90545 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707633, fieldID:1 I20240629 17:26:12.502260 90623 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450768056432707633 I20240629 17:26:12.502305 90404 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707633, fieldID:103 I20240629 17:26:12.502261 90615 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450768056432707633 I20240629 17:26:12.502482 90631 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707633, fieldID:102 I20240629 17:26:12.502503 90586 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707633, fieldID:104 I20240629 17:26:12.502523 90623 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707633, fieldID:101 I20240629 17:26:12.502593 90615 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707633, fieldID:0 I20240629 17:26:12.502624 90629 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450768056432707633, fieldID:100 I20240629 17:26:12.612509 90581 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707632, fieldID:0 I20240629 17:26:12.630728 90537 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707632, fieldID:103 I20240629 17:26:12.636991 90404 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707633, fieldID:103 I20240629 17:26:12.638721 90621 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707632, fieldID:101 I20240629 17:26:12.640225 90592 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707634, fieldID:0 I20240629 17:26:12.643106 90616 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707634, fieldID:103 I20240629 17:26:12.644737 90586 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707633, fieldID:104 I20240629 17:26:12.646517 90601 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707632, fieldID:1 I20240629 17:26:12.650631 90448 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707634, fieldID:104 I20240629 17:26:12.652516 90545 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707633, fieldID:1 I20240629 17:26:12.653824 90626 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707632, fieldID:104 I20240629 17:26:12.658324 90452 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707634, fieldID:101 I20240629 17:26:12.664736 90622 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707634, fieldID:100 I20240629 17:26:12.775712 90623 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707633, fieldID:101 I20240629 17:26:12.784487 90597 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707634, fieldID:1 I20240629 17:26:12.804821 90615 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707633, fieldID:0 I20240629 17:26:12.865456 90629 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707633, fieldID:100 I20240629 17:26:13.126859 90591 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707632, fieldID:100 I20240629 17:26:13.167575 90633 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:26:13.167642 90633 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:26:13.172920 90631 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:26:13.173005 90631 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:26:13.207057 90634 IndexConfigGenerator.cpp:32] [SEGCORE][VecIndexConfig][milvus] VecIndexConfig: origin_indextype:IVF_SQ8 indextype: IVF_FLAT_CC metrictype: IP I20240629 17:26:13.207124 90634 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_FLAT_CC with version 1 I20240629 17:26:14.604280 90631 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1431.185623 ms) I20240629 17:26:14.607739 90634 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1400.542075 ms) I20240629 17:26:14.608099 90631 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707633, fieldID:102 I20240629 17:26:14.612417 90634 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707632, fieldID:102 I20240629 17:26:14.612694 90633 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1444.933103 ms) I20240629 17:26:14.616132 90633 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450768056432707634, fieldID:102 I20240629 17:26:31.982033 90695 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:26:31.982095 90695 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 3 I20240629 17:26:31.982115 90695 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:26:32.088847 90695 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:26:32.088904 90695 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:26:32.088915 90695 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:26:32.097658 90695 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (8.656553 ms) I20240629 17:26:32.097723 90695 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:26:52.038803 90583 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:26:52.038861 90583 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 3 I20240629 17:26:52.038877 90583 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:26:52.155416 90583 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:26:52.155472 90583 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:26:52.155483 90583 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:26:52.163748 90583 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (8.214769 ms) I20240629 17:26:52.163797 90583 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:27:01.982738 90743 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:27:01.982827 90743 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 3 I20240629 17:27:01.982862 90743 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:27:02.117470 90743 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:27:02.117532 90743 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:27:02.117544 90743 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:27:02.128866 90743 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (11.277173 ms) I20240629 17:27:02.128938 90743 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:27:12.507318 90729 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1220.914924 ms) I20240629 17:27:12.583914 90702 factory.cc:20] [KNOWHERE][Create][milvus] create knowhere index IVF_SQ8 with version 1 I20240629 17:27:12.583984 90702 VectorMemIndex.cpp:124] [SEGCORE][Load][milvus] load index files: 7 I20240629 17:27:12.584000 90702 VectorMemIndex.cpp:141] [SEGCORE][Load][milvus] load with slice meta: 1 I20240629 17:27:12.809864 90583 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Build index: done (1327.555728 ms) I20240629 17:27:12.890282 90702 VectorMemIndex.cpp:205] [SEGCORE][Load][milvus] construct binary set... I20240629 17:27:12.890342 90702 VectorMemIndex.cpp:208] [SEGCORE][Load][milvus] add index data to binary set: IVF_SQ8 I20240629 17:27:12.890353 90702 VectorMemIndex.cpp:216] [SEGCORE][Load][milvus] load index into Knowhere... I20240629 17:27:12.953841 90702 time_recorder.cc:49] [KNOWHERE][PrintTimeRecord][milvus] Load index: done (63.373117 ms) I20240629 17:27:12.954092 90702 VectorMemIndex.cpp:218] [SEGCORE][Load][milvus] load vector index done I20240629 17:27:13.207113 90614 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 101 of segment 450767622823027923 I20240629 17:27:13.207114 90712 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 1 of segment 450767622823027923 I20240629 17:27:13.207229 90712 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027923, fieldID:1 I20240629 17:27:13.207114 90705 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 0 of segment 450767622823027923 I20240629 17:27:13.207195 90614 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027923, fieldID:101 I20240629 17:27:13.207293 90705 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027923, fieldID:0 I20240629 17:27:13.207192 90732 SegmentSealedImpl.cpp:198] [SEGCORE][LoadFieldData][milvus] start to load field data 100 of segment 450767622823027923 I20240629 17:27:13.207469 90732 SegmentSealedImpl.cpp:207] [SEGCORE][LoadFieldData][milvus] finish submitting LoadFieldDatasFromRemote task to thread pool, segmentID:450767622823027923, fieldID:100 I20240629 17:27:13.239930 90712 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027923, fieldID:1 I20240629 17:27:13.332057 90732 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027923, fieldID:100 I20240629 17:27:13.340961 90614 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027923, fieldID:101 I20240629 17:27:13.356076 90705 SegmentSealedImpl.cpp:217] [SEGCORE][LoadFieldData][milvus] finish loading segment field, segmentID:450767622823027923, fieldID:0

lzhin commented 5 months ago

datanode: [2024/06/28 10:43:44.923 +08:00] [INFO] [datanode/service.go:296] ["current DataNode state"] [state=Initializing] [2024/06/28 10:43:44.923 +08:00] [INFO] [datanode/service.go:191] ["DataNode gRPC services successfully initialized"] [2024/06/28 10:43:44.923 +08:00] [INFO] [datanode/data_node.go:322] ["start id allocator done"] [role=datanode] [2024/06/28 10:43:44.927 +08:00] [INFO] [storage/remote_chunk_manager.go:83] ["remote chunk manager init success."] [remote=aws] [bucketname=a-bucket] [root=files] [2024/06/28 10:43:44.927 +08:00] [INFO] [datanode/data_node.go:301] ["DataNode Background GC Start"] [2024/06/28 10:43:44.927 +08:00] [DEBUG] [sessionutil/session_util.go:432] ["service begin to register to etcd"] [serverName=datanode] [ServerID=664] [2024/06/28 10:43:44.927 +08:00] [INFO] [datanode/event_manager.go:48] ["Start watch channel"] [prefix=channelwatch/664] [2024/06/28 10:43:44.929 +08:00] [INFO] [sessionutil/session_util.go:462] ["put session key into etcd"] [key=by-dev/meta/session/datanode-664] [value="{\"ServerID\":664,\"ServerName\":\"datanode\",\"Address\":\"10.137.47.187:21124\",\"TriggerKill\":true,\"Version\":\"2.3.5\",\"IndexEngineVersion\":{},\"LeaseID\":4772566873342794814,\"HostName\":\"audio-feature-extraction-datanode-mky-967131-zf48t.hy\"}"] [2024/06/28 10:43:44.929 +08:00] [INFO] [sessionutil/session_util.go:472] ["Service registered successfully"] [ServerName=datanode] [serverID=664] [2024/06/28 10:43:44.929 +08:00] [INFO] [datanode/data_node.go:187] ["DataNode Register Finished"] [2024/06/28 10:43:44.930 +08:00] [INFO] [datanode/service.go:196] ["DataNode gRPC services successfully started"] [2024/06/28 10:43:44.930 +08:00] [DEBUG] [components/data_node.go:57] ["Datanode successfully started"] INFO[0000] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0000] TCP connection established local_addr="10.137.47.187:41536" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0000] Connection is ready local_addr="10.137.47.187:41536" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0000] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0000] TCP connection established local_addr="10.137.47.187:41540" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0000] Connection is ready local_addr="10.137.47.187:41540" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0000] Connected consumer consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[0000] Created consumer consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[0000] Broker notification of Closed consumer: 1 local_addr="10.137.47.187:41540" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0000] Reconnecting to broker in 104.363277ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[0000] Connected consumer consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[0000] Reconnected consumer to broker consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[0000] Connected consumer consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[0000] Created consumer consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[0000] Broker notification of Closed consumer: 2 local_addr="10.137.47.187:41540" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0000] Reconnecting to broker in 119.297043ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[0000] Connected consumer consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[0000] Reconnected consumer to broker consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[0000] Connected consumer consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[0000] Created consumer consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[0000] Broker notification of Closed consumer: 3 local_addr="10.137.47.187:41540" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0000] Reconnecting to broker in 102.878446ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[0000] Connected consumer consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[0000] Reconnected consumer to broker consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[0000] Connected consumer consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[0000] Created consumer consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[0000] Broker notification of Closed consumer: 4 local_addr="10.137.47.187:41540" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0000] Reconnecting to broker in 110.953562ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[0000] Connected consumer consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[0000] Reconnected consumer to broker consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[0000] Connected consumer consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[0000] Created consumer consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[0000] Broker notification of Closed consumer: 5 local_addr="10.137.47.187:41540" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0000] Reconnecting to broker in 114.456582ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[0001] Connected consumer consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[0001] Reconnected consumer to broker consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[0025] Connected consumer consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[0025] Created consumer consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[0025] Broker notification of Closed consumer: 6 local_addr="10.137.47.187:41540" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0025] Reconnecting to broker in 108.7811ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[0025] Connected consumer consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[0025] Reconnected consumer to broker consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[0025] Connected consumer consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0025] Created consumer consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0025] Broker notification of Closed consumer: 7 local_addr="10.137.47.187:41540" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0025] Reconnecting to broker in 116.425204ms consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0025] Connected consumer consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0025] Reconnected consumer to broker consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0025] Connected consumer consumerID=8 name=vzcsw subscription=datanode-664-by-dev-rootcoord-dml_4_450684035915636293v2-false topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0025] Created consumer consumerID=8 name=vzcsw subscription=datanode-664-by-dev-rootcoord-dml_4_450684035915636293v2-false topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0025] Broker notification of Closed consumer: 8 local_addr="10.137.47.187:41540" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[0025] Reconnecting to broker in 105.15793ms consumerID=8 name=vzcsw subscription=datanode-664-by-dev-rootcoord-dml_4_450684035915636293v2-false topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0025] Connected consumer consumerID=8 name=vzcsw subscription=datanode-664-by-dev-rootcoord-dml_4_450684035915636293v2-false topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0025] Reconnected consumer to broker consumerID=8 name=vzcsw subscription=datanode-664-by-dev-rootcoord-dml_4_450684035915636293v2-false topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0026] The consumer[8] successfully unsubscribed consumerID=8 name=vzcsw subscription=datanode-664-by-dev-rootcoord-dml_4_450684035915636293v2-false topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0026] Closing consumer=8 consumerID=8 name=vzcsw subscription=datanode-664-by-dev-rootcoord-dml_4_450684035915636293v2-false topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0026] Closed consumer consumerID=8 name=vzcsw subscription=datanode-664-by-dev-rootcoord-dml_4_450684035915636293v2-false topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[0026] close consumer, exit reconnect consumerID=8 name=vzcsw subscription=datanode-664-by-dev-rootcoord-dml_4_450684035915636293v2-false topic="persistent://public/default/by-dev-rootcoord-dml_4" [2024/06/29 11:33:55.987 +08:00] [ERROR] [retry/retry.go:46] ["retry func failed"] ["retry time"=0] [error="segment not found during flush: segment not found[segment=450768056354239492]: unrecoverable error"] [stack="github.com/milvus-io/milvus/pkg/util/retry.Do\n\t/milvus/pkg/util/retry/retry.go:46\ngithub.com/milvus-io/milvus/internal/datanode.(insertBufferNode).Sync\n\t/milvus/internal/datanode/flow_graph_insert_buffer_node.go:482\ngithub.com/milvus-io/milvus/internal/datanode.(insertBufferNode).Operate\n\t/milvus/internal/datanode/flow_graph_insert_buffer_node.go:207\ngithub.com/milvus-io/milvus/internal/util/flowgraph.(*nodeCtxManager).workNodeStart\n\t/milvus/internal/util/flowgraph/node.go:125"] INFO[90224] Reconnecting to broker in 117.77225ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90224] Reconnecting to broker in 118.966076ms consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90224] Reconnecting to broker in 105.894418ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90224] Reconnecting to broker in 111.271466ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90224] Reconnecting to broker in 118.082357ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90224] Reconnecting to broker in 105.153737ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90224] Reconnecting to broker in 110.648951ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90224] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90224] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90224] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90224] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90224] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90224] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90224] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90224] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90225] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90225] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90226] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90226] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90226] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90226] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90226] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90226] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90226] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90226] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90226] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90226] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90226] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90226] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90226] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90226] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90227] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90227] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90227] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90227] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90227] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90227] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90228] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90228] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90228] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90228] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90228] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90228] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90228] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90228] Failed to connect to broker. error="dial tcp 10.137.100.169:6650: connect: connection refused" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90231] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90231] TCP connection established local_addr="10.137.47.187:39622" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90231] Connection is ready local_addr="10.137.47.187:39622" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90231] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90231] Reconnecting to broker in 117.250593ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90231] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90231] Reconnecting to broker in 106.230261ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90231] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90231] Reconnecting to broker in 112.618749ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90231] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90231] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90231] Reconnecting to broker in 109.791245ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90231] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90231] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90231] Reconnecting to broker in 116.323502ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90231] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90231] Reconnecting to broker in 113.026469ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90231] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90231] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90231] Reconnecting to broker in 104.905185ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90231] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90231] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90231] Reconnecting to broker in 103.683202ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90231] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90231] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90231] Reconnecting to broker in 100.643957ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90231] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90231] Reconnecting to broker in 108.105702ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90231] Failed to lookup topic consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" ERRO[90231] Failed to create consumer at reconnect consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90231] Reconnecting to broker in 115.781678ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90231] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90231] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90231] Reconnecting to broker in 107.005577ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90231] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90231] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90231] Reconnecting to broker in 102.062997ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90231] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90231] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90231] Reconnecting to broker in 118.109571ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90231] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90231] Reconnecting to broker in 116.151987ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90231] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90231] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90231] Reconnecting to broker in 107.64844ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90231] Failed to lookup topic consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" ERRO[90231] Failed to create consumer at reconnect consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90231] Reconnecting to broker in 112.863035ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90231] Failed to lookup topic consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" ERRO[90231] Failed to create consumer at reconnect consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90231] Reconnecting to broker in 117.011151ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90231] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90231] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90231] Reconnecting to broker in 116.045073ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90231] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90231] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90231] Reconnecting to broker in 101.244733ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90231] Failed to lookup topic consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" ERRO[90231] Failed to create consumer at reconnect consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90231] Reconnecting to broker in 111.028953ms consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90231] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90231] Reconnecting to broker in 101.087909ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90231] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90231] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90231] Reconnecting to broker in 109.113108ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90231] Failed to lookup topic consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" ERRO[90231] Failed to create consumer at reconnect consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90231] Reconnecting to broker in 104.572916ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90231] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90231] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90231] Reconnecting to broker in 100.800397ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90231] Failed to lookup topic consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" ERRO[90231] Failed to create consumer at reconnect consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90231] Reconnecting to broker in 101.876709ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90231] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90231] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90231] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90231] Reconnecting to broker in 108.194165ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90232] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90232] Reconnecting to broker in 115.31364ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90232] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90232] Reconnecting to broker in 110.533066ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" ERRO[90232] Failed to create consumer at reconnect consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90232] Reconnecting to broker in 116.35993ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" ERRO[90232] Failed to create consumer at reconnect consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90232] Reconnecting to broker in 117.949926ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" ERRO[90232] Failed to create consumer at reconnect consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90232] Reconnecting to broker in 105.524708ms consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90232] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90232] Reconnecting to broker in 111.994605ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90232] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90232] Reconnecting to broker in 107.211209ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" ERRO[90232] Failed to create consumer at reconnect consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90232] Reconnecting to broker in 103.154218ms consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90232] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90232] Reconnecting to broker in 119.225631ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90232] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90232] Reconnecting to broker in 115.695992ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90232] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90232] Reconnecting to broker in 111.33871ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" ERRO[90232] Failed to create consumer at reconnect consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90232] Reconnecting to broker in 103.778351ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" ERRO[90232] Failed to create consumer at reconnect consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90232] Reconnecting to broker in 101.380901ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90232] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90232] Reconnecting to broker in 114.770927ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" ERRO[90232] Failed to create consumer at reconnect consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90232] Reconnecting to broker in 112.057441ms consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" ERRO[90232] Failed to create consumer at reconnect consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" ERRO[90232] Failed to create consumer at reconnect consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90232] Reconnecting to broker in 112.083299ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90232] Reconnecting to broker in 106.292573ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90232] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90232] Reconnecting to broker in 106.678703ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90232] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90232] Reconnecting to broker in 115.909057ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90232] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90232] Reconnecting to broker in 110.528233ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90232] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90232] Reconnecting to broker in 118.373086ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" ERRO[90232] Failed to create consumer at reconnect consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90232] Reconnecting to broker in 118.081739ms consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" ERRO[90232] Failed to create consumer at reconnect consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90232] Reconnecting to broker in 112.42127ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90232] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90232] Reconnecting to broker in 100.657805ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90232] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90232] Reconnecting to broker in 119.670735ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90232] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90232] Reconnecting to broker in 118.286306ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" ERRO[90232] Failed to create consumer at reconnect consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90232] Reconnecting to broker in 112.57968ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90232] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90232] Reconnecting to broker in 108.149836ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90232] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90232] Reconnecting to broker in 103.67582ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" ERRO[90232] Failed to create consumer at reconnect consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90232] Reconnecting to broker in 116.195836ms consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" ERRO[90232] Failed to create consumer at reconnect consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90232] Reconnecting to broker in 103.890421ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90232] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90232] Reconnecting to broker in 102.062382ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90232] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90232] Reconnecting to broker in 108.413983ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" ERRO[90232] Failed to create consumer at reconnect consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90232] Reconnecting to broker in 115.621177ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90232] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90232] Reconnecting to broker in 117.334626ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90232] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90232] Reconnecting to broker in 112.229796ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90232] Failed to create consumer at reconnect consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90232] Reconnecting to broker in 112.283662ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90232] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90232] Reconnecting to broker in 119.43796ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90232] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90232] Reconnecting to broker in 118.85889ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" ERRO[90232] Failed to create consumer at reconnect consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90232] Reconnecting to broker in 101.558562ms consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" ERRO[90232] Failed to create consumer at reconnect consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90232] Reconnecting to broker in 117.297754ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90232] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90232] Reconnecting to broker in 111.849113ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90232] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90232] Reconnecting to broker in 103.510675ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" ERRO[90232] Failed to create consumer at reconnect consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90232] Reconnecting to broker in 105.059381ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" ERRO[90232] Failed to create consumer at reconnect consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90232] Reconnecting to broker in 114.751944ms consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90232] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90232] Reconnecting to broker in 106.893457ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90232] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90232] Reconnecting to broker in 100.119442ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90232] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90232] Reconnecting to broker in 106.844543ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" ERRO[90232] Failed to create consumer at reconnect consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90232] Reconnecting to broker in 105.644625ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90232] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90232] Reconnecting to broker in 112.86614ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" ERRO[90232] Failed to create consumer at reconnect consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90232] Reconnecting to broker in 114.053463ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" ERRO[90232] Failed to create consumer at reconnect consumerID=5 error=MetadataError name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90232] Reconnecting to broker in 114.091303ms consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" ERRO[90232] Failed to create consumer at reconnect consumerID=7 error=MetadataError name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90232] Reconnecting to broker in 114.634315ms consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" ERRO[90232] Failed to create consumer at reconnect consumerID=3 error=MetadataError name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90232] Reconnecting to broker in 116.671111ms consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" ERRO[90232] Failed to create consumer at reconnect consumerID=6 error=MetadataError name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90232] Reconnecting to broker in 103.883491ms consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" ERRO[90232] Failed to create consumer at reconnect consumerID=2 error=MetadataError name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90232] Reconnecting to broker in 106.142956ms consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" ERRO[90232] Failed to create consumer at reconnect consumerID=4 error=MetadataError name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90232] Reconnecting to broker in 100.665202ms consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" WARN[90232] Failed to lookup topic error=MetadataError message="org.apache.pulsar.broker.web.RestException: Policies not found for public/default namespace" serviceURL="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" topic="persistent://public/default/by-dev-rootcoord-dml_15" WARN[90232] Failed to lookup topic consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" ERRO[90232] Failed to create consumer at reconnect consumerID=1 error=MetadataError name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90232] Reconnecting to broker in 115.330158ms consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90233] Connecting to broker remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90233] TCP connection established local_addr="10.137.47.187:39742" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90233] Connection is ready local_addr="10.137.47.187:39742" remote_addr="pulsar://huyalive-skyeye-audio-pulsar.huya.info:6650" INFO[90233] Connected consumer consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90233] Reconnected consumer to broker consumerID=5 name=huvbv subscription=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true topic="persistent://public/default/by-dev-rootcoord-dml_8" INFO[90233] Connected consumer consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90233] Reconnected consumer to broker consumerID=4 name=hdkru subscription=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true topic="persistent://public/default/by-dev-rootcoord-dml_5" INFO[90233] Connected consumer consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90233] Reconnected consumer to broker consumerID=1 name=jyvmy subscription=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true topic="persistent://public/default/by-dev-rootcoord-dml_15" INFO[90233] Connected consumer consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90233] Reconnected consumer to broker consumerID=3 name=jhzrz subscription=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true topic="persistent://public/default/by-dev-rootcoord-dml_2" INFO[90233] Connected consumer consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90233] Reconnected consumer to broker consumerID=7 name=ostmt subscription=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true topic="persistent://public/default/by-dev-rootcoord-dml_4" INFO[90233] Connected consumer consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90233] Reconnected consumer to broker consumerID=6 name=xosvk subscription=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true topic="persistent://public/default/by-dev-rootcoord-dml_10" INFO[90233] Connected consumer consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3" INFO[90233] Reconnected consumer to broker consumerID=2 name=bdxwh subscription=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true topic="persistent://public/default/by-dev-rootcoord-dml_3"

xiaofan-luan commented 5 months ago

@lzhin what happened to this pulsar cluster?

I saw error logs Policies not found for public/default namespace and Failed to connect to broker. That means the pulsar is not working correctly

lzhin commented 5 months ago

@lzhin what happened to this pulsar cluster?

I saw error logs Policies not found for public/default namespace and Failed to connect to broker. That means the pulsar is not working correctly

I dont know . this is the pulsar log: 07:49:11.774 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_8] [datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true] Updating cursor info ledgerId=1779 mark-delete=1731:20710 07:49:11.774 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_7] [datanode-663-by-dev-rootcoord-dml_7_450684035962993998v2-true] Updating cursor info ledgerId=1782 mark-delete=1723:20710 07:49:11.774 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_9] [datanode-663-by-dev-rootcoord-dml_9_450684035915233807v1-true] Updating cursor info ledgerId=1776 mark-delete=1722:20710 07:49:11.775 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_0] Updated cursor querynode-636-by-dev-rootcoord-dml_0_450684035915234141v1-true with ledger id 1770 md-position=1718:47387 rd-position=1718:47388 07:49:11.775 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_13] Updated cursor querynode-633-by-dev-rootcoord-dml_13_450684035915233981v2-true with ledger id 1773 md-position=1730:20710 rd-position=1730:20711 07:49:11.775 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_13] Updated cursor datanode-663-by-dev-rootcoord-dml_13_450684035915233981v2-true with ledger id 1781 md-position=1730:20710 rd-position=1730:20711 07:49:11.776 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_8] Updated cursor querynode-632-by-dev-rootcoord-dml_8_450684035887223084v2-true with ledger id 1777 md-position=1731:20710 rd-position=1731:20711 07:49:11.776 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_14] Updated cursor datanode-663-by-dev-rootcoord-dml_14_450684035915234141v0-true with ledger id 1772 md-position=1728:20710 rd-position=1728:20711 07:49:11.776 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_6] Updated cursor querynode-632-by-dev-rootcoord-dml_6_450684035962993998v1-true with ledger id 1774 md-position=1729:20710 rd-position=1729:20711 07:49:11.776 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_0] Updated cursor querynode-634-by-dev-rootcoord-dml_0_450684035915234141v1-true with ledger id 1771 md-position=1718:47387 rd-position=1718:47388 07:49:11.776 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_6] Updated cursor datanode-663-by-dev-rootcoord-dml_6_450684035887223084v0-true with ledger id 1778 md-position=1729:20710 rd-position=1729:20711 07:49:11.776 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_11] Updated cursor querynode-633-by-dev-rootcoord-dml_11_450684035915233981v0-true with ledger id 1775 md-position=1724:20710 rd-position=1724:20711 07:49:11.776 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_15] Updated cursor querynode-636-by-dev-rootcoord-dml_15_450684035915233807v0-true with ledger id 1780 md-position=1725:20710 rd-position=1725:20711 07:49:11.776 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_8] Updated cursor datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true with ledger id 1779 md-position=1731:20710 rd-position=1731:20711 07:49:11.776 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_7] Updated cursor datanode-663-by-dev-rootcoord-dml_7_450684035962993998v2-true with ledger id 1782 md-position=1723:20710 rd-position=1723:20711 07:49:11.776 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_9] Updated cursor datanode-663-by-dev-rootcoord-dml_9_450684035915233807v1-true with ledger id 1776 md-position=1722:20710 rd-position=1722:20711 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_0][querynode-636-by-dev-rootcoord-dml_0_450684035915234141v1-true] Successfully closed & deleted ledger 1659 in cursor 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_13][querynode-633-by-dev-rootcoord-dml_13_450684035915233981v2-true] Successfully closed & deleted ledger 1662 in cursor 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_13][datanode-663-by-dev-rootcoord-dml_13_450684035915233981v2-true] Successfully closed & deleted ledger 1661 in cursor 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_8][querynode-632-by-dev-rootcoord-dml_8_450684035887223084v2-true] Successfully closed & deleted ledger 1660 in cursor 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_14][datanode-663-by-dev-rootcoord-dml_14_450684035915234141v0-true] Successfully closed & deleted ledger 1664 in cursor 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_6][querynode-632-by-dev-rootcoord-dml_6_450684035962993998v1-true] Successfully closed & deleted ledger 1665 in cursor 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_0][querynode-634-by-dev-rootcoord-dml_0_450684035915234141v1-true] Successfully closed & deleted ledger 1653 in cursor 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_6][datanode-663-by-dev-rootcoord-dml_6_450684035887223084v0-true] Successfully closed & deleted ledger 1673 in cursor 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_11][querynode-633-by-dev-rootcoord-dml_11_450684035915233981v0-true] Successfully closed & deleted ledger 1655 in cursor 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_8][datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true] Successfully closed & deleted ledger 1666 in cursor 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_15][querynode-636-by-dev-rootcoord-dml_15_450684035915233807v0-true] Successfully closed & deleted ledger 1676 in cursor 07:49:11.777 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_7][datanode-663-by-dev-rootcoord-dml_7_450684035962993998v2-true] Successfully closed & deleted ledger 1668 in cursor 07:49:11.778 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_9][datanode-663-by-dev-rootcoord-dml_9_450684035915233807v1-true] Successfully closed & deleted ledger 1674 in cursor 07:49:21.758 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_9] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 57], pulsar/cursor=[113, 117, 101, 114, 121, 110, 111, 100, 101, 45, 54, 51, 51, 45, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 57, 95, 52, 53, 48, 54, 56, 52, 48, 51, 53, 57, 49, 53, 50, 51, 51, 56, 48, 55, 118, 49, 45, 116, 114, 117, 101], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:49:21.758 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_7] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 55], pulsar/cursor=[113, 117, 101, 114, 121, 110, 111, 100, 101, 45, 54, 51, 50, 45, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 55, 95, 52, 53, 48, 54, 56, 52, 48, 51, 53, 57, 54, 50, 57, 57, 51, 57, 57, 56, 118, 50, 45, 116, 114, 117, 101], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:49:21.758 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_7] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 55], pulsar/cursor=[113, 117, 101, 114, 121, 110, 111, 100, 101, 45, 54, 51, 51, 45, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 55, 95, 52, 53, 48, 54, 56, 52, 48, 51, 53, 57, 54, 50, 57, 57, 51, 57, 57, 56, 118, 50, 45, 116, 114, 117, 101], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:49:21.758 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_9] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 57], pulsar/cursor=[113, 117, 101, 114, 121, 110, 111, 100, 101, 45, 54, 51, 52, 45, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 57, 95, 52, 53, 48, 54, 56, 52, 48, 51, 53, 57, 49, 53, 50, 51, 51, 56, 48, 55, 118, 49, 45, 116, 114, 117, 101], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:49:21.758 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_10] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 48], pulsar/cursor=[100, 97, 116, 97, 110, 111, 100, 101, 45, 54, 54, 52, 45, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 48, 95, 52, 53, 48, 54, 56, 52, 48, 51, 53, 57, 49, 53, 50, 51, 51, 56, 48, 55, 118, 50, 45, 116, 114, 117, 101], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:49:21.758 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_14] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 52], pulsar/cursor=[113, 117, 101, 114, 121, 110, 111, 100, 101, 45, 54, 51, 50, 45, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 52, 95, 52, 53, 48, 54, 56, 52, 48, 51, 53, 57, 49, 53, 50, 51, 52, 49, 52, 49, 118, 48, 45, 116, 114, 117, 101], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:49:21.758 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_11] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 49], pulsar/cursor=[113, 117, 101, 114, 121, 110, 111, 100, 101, 45, 54, 51, 52, 45, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 49, 95, 52, 53, 48, 54, 56, 52, 48, 51, 53, 57, 49, 53, 50, 51, 51, 57, 56, 49, 118, 48, 45, 116, 114, 117, 101], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:49:21.758 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_13] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 51], pulsar/cursor=[113, 117, 101, 114, 121, 110, 111, 100, 101, 45, 54, 51, 50, 45, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 51, 95, 52, 53, 48, 54, 56, 52, 48, 51, 53, 57, 49, 53, 50, 51, 51, 57, 56, 49, 118, 50, 45, 116, 114, 117, 101], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:49:21.758 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_7] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 55], pulsar/cursor=[100, 97, 116, 97, 110, 111, 100, 101, 45, 54, 54, 50, 45, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 55, 95, 52, 53, 48, 54, 56, 52, 48, 51, 53, 56, 56, 55, 50, 50, 51, 48, 56, 52, 118, 49, 45, 116, 114, 117, 101], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:49:21.758 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_10] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 48], pulsar/cursor=[113, 117, 101, 114, 121, 110, 111, 100, 101, 45, 54, 51, 51, 45, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 48, 95, 52, 53, 48, 54, 56, 52, 48, 51, 53, 57, 49, 53, 50, 51, 51, 56, 48, 55, 118, 50, 45, 116, 114, 117, 101], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:49:21.761 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1783 07:49:21.762 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1784 07:49:21.762 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1785 07:49:21.763 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1786 07:49:21.763 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1787 07:49:21.763 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1788 07:49:21.763 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1789 07:49:21.763 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1790 07:49:21.763 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_7] [querynode-632-by-dev-rootcoord-dml_7_450684035962993998v2-true] Updating cursor info ledgerId=1784 mark-delete=1723:20760 07:49:21.763 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_9] [querynode-633-by-dev-rootcoord-dml_9_450684035915233807v1-true] Updating cursor info ledgerId=1783 mark-delete=1722:20760 07:49:21.763 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1791 07:49:21.763 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1792 07:49:21.764 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_14] [querynode-632-by-dev-rootcoord-dml_14_450684035915234141v0-true] Updating cursor info ledgerId=1788 mark-delete=1728:20760 07:49:21.764 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_7] [datanode-662-by-dev-rootcoord-dml_7_450684035887223084v1-true] Updating cursor info ledgerId=1791 mark-delete=1723:20760 07:49:21.764 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_10] [querynode-633-by-dev-rootcoord-dml_10_450684035915233807v2-true] Updating cursor info ledgerId=1792 mark-delete=1726:20760 07:49:21.764 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_7] [querynode-633-by-dev-rootcoord-dml_7_450684035962993998v2-true] Updating cursor info ledgerId=1785 mark-delete=1723:20760 07:49:21.764 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_13] [querynode-632-by-dev-rootcoord-dml_13_450684035915233981v2-true] Updating cursor info ledgerId=1790 mark-delete=1730:20760 07:49:21.764 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_10] [datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true] Updating cursor info ledgerId=1787 mark-delete=1726:20760 07:49:21.764 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_11] [querynode-634-by-dev-rootcoord-dml_11_450684035915233981v0-true] Updating cursor info ledgerId=1789 mark-delete=1724:20760 07:49:21.764 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImpl - [public/default/persistent/by-dev-rootcoord-dml_9] [querynode-634-by-dev-rootcoord-dml_9_450684035915233807v1-true] Updating cursor info ledgerId=1786 mark-delete=1722:20760 07:49:21.764 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_7] Updated cursor querynode-632-by-dev-rootcoord-dml_7_450684035962993998v2-true with ledger id 1784 md-position=1723:20760 rd-position=1723:20761 07:49:21.765 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_9] Updated cursor querynode-633-by-dev-rootcoord-dml_9_450684035915233807v1-true with ledger id 1783 md-position=1722:20760 rd-position=1722:20761 07:49:21.765 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_14] Updated cursor querynode-632-by-dev-rootcoord-dml_14_450684035915234141v0-true with ledger id 1788 md-position=1728:20760 rd-position=1728:20761 07:49:21.765 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_7] Updated cursor datanode-662-by-dev-rootcoord-dml_7_450684035887223084v1-true with ledger id 1791 md-position=1723:20760 rd-position=1723:20761 07:49:21.765 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_10] Updated cursor querynode-633-by-dev-rootcoord-dml_10_450684035915233807v2-true with ledger id 1792 md-position=1726:20760 rd-position=1726:20761 07:49:21.765 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_7] Updated cursor querynode-633-by-dev-rootcoord-dml_7_450684035962993998v2-true with ledger id 1785 md-position=1723:20760 rd-position=1723:20761 07:49:21.765 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_13] Updated cursor querynode-632-by-dev-rootcoord-dml_13_450684035915233981v2-true with ledger id 1790 md-position=1730:20760 rd-position=1730:20761 07:49:21.765 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_9] Updated cursor querynode-634-by-dev-rootcoord-dml_9_450684035915233807v1-true with ledger id 1786 md-position=1722:20760 rd-position=1722:20761 07:49:21.765 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_10] Updated cursor datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true with ledger id 1787 md-position=1726:20760 rd-position=1726:20761 07:49:21.765 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_11] Updated cursor querynode-634-by-dev-rootcoord-dml_11_450684035915233981v0-true with ledger id 1789 md-position=1724:20760 rd-position=1724:20761 07:49:21.767 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_7][querynode-632-by-dev-rootcoord-dml_7_450684035962993998v2-true] Successfully closed & deleted ledger 1696 in cursor 07:49:21.767 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_9][querynode-633-by-dev-rootcoord-dml_9_450684035915233807v1-true] Successfully closed & deleted ledger 1693 in cursor 07:49:21.767 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_7][datanode-662-by-dev-rootcoord-dml_7_450684035887223084v1-true] Successfully closed & deleted ledger 1697 in cursor 07:49:21.767 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_14][querynode-632-by-dev-rootcoord-dml_14_450684035915234141v0-true] Successfully closed & deleted ledger 1694 in cursor 07:49:21.767 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_10][querynode-633-by-dev-rootcoord-dml_10_450684035915233807v2-true] Successfully closed & deleted ledger 1690 in cursor 07:49:21.767 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_7][querynode-633-by-dev-rootcoord-dml_7_450684035962993998v2-true] Successfully closed & deleted ledger 1691 in cursor 07:49:21.767 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_9][querynode-634-by-dev-rootcoord-dml_9_450684035915233807v1-true] Successfully closed & deleted ledger 1692 in cursor 07:49:21.767 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_13][querynode-632-by-dev-rootcoord-dml_13_450684035915233981v2-true] Successfully closed & deleted ledger 1698 in cursor 07:49:21.767 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_10][datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true] Successfully closed & deleted ledger 1695 in cursor 07:49:21.767 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [public/default/persistent/by-dev-rootcoord-dml_11][querynode-634-by-dev-rootcoord-dml_11_450684035915233981v0-true] Successfully closed & deleted ledger 1689 in cursor 07:50:33.227 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true 07:50:33.227 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 360 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=360, logFile=data/standalone/bookkeeper0/current/168.log, ledgerIdAssigned=-1}]. 07:50:33.233 [pool-6-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/16a.log for logId 362. 07:50:33.364 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerForSingleEntryLog - Synced entry logger 360 to disk. 07:52:33.213 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Extracting entry log meta from entryLogId: 360 07:52:33.442 [db-storage-cleanup-10-1] INFO org.apache.bookkeeper.bookie.storage.ldb.EntryLocationIndex - Deleting indexes for ledgers: [1664, 1665, 1666, 1667, 1668, 1669, 1670, 1671, 1672, 1673, 1674, 1675, 1676, 1677, 1678, 1679, 1680, 1681, 1682, 1683, 1684, 1685, 1686, 1687, 1688, 1689, 1690, 1691, 1692, 1693, 1694, 1695, 1696, 1697, 1698, 1641, 1642, 1643, 1644, 1645, 1646, 1647, 1648, 1649, 1650, 1651, 1652, 1653, 1654, 1655, 1656, 1657, 1658, 1659, 1660, 1661, 1662, 1663] 07:52:33.510 [db-storage-cleanup-10-1] INFO org.apache.bookkeeper.bookie.storage.ldb.EntryLocationIndex - Deleted indexes for 83605 entries from 58 ledgers in 0.067 seconds 07:53:33.938 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9904 xid:136492 sent:136492 recv:137592 queuedpkts:0 pendingresp:0 queuedevents:1] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_5 07:53:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] End TrimConsumedLedgers. ledgers=13 totalSize=8604797380 07:53:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] End TrimConsumedLedgers. ledgers=13 totalSize=8602253108 07:53:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Removing ledger 1533 - size: 791970502 07:53:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] End TrimConsumedLedgers. ledgers=13 totalSize=8601669039 07:53:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Removing ledger 1534 - size: 792682369 07:53:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Removing ledger 1535 - size: 793847829 07:53:33.938 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9904 xid:136492 sent:136492 recv:137593 queuedpkts:0 pendingresp:0 queuedevents:1] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_4 07:53:33.938 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9904 xid:136492 sent:136492 recv:137593 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_3 07:53:59.028 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_5] Closing ledger 1732 for being full 07:53:59.030 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Creating a new ledger 07:53:59.030 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 53], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:53:59.032 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1793 07:53:59.032 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Created new ledger 1793 07:53:59.033 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9910 xid:136509 sent:136509 recv:137611 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_5 07:53:59.033 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=datanode-662-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1732:49999, readPos=1732:50000} to 1793:-1 since ledger consumed completely 07:53:59.033 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true, ackPos=1732:49999, readPos=1732:50000} to 1793:-1 since ledger consumed completely 07:53:59.034 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-633-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1732:49999, readPos=1732:50000} to 1793:-1 since ledger consumed completely 07:53:59.034 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-634-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1732:49999, readPos=1732:50000} to 1793:-1 since ledger consumed completely 07:53:59.034 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-636-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1732:49999, readPos=1732:50000} to 1793:-1 since ledger consumed completely 07:53:59.556 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_3] Closing ledger 1733 for being full 07:53:59.565 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Creating a new ledger 07:53:59.565 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 51], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:53:59.567 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1794 07:53:59.567 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Created new ledger 1794 07:53:59.569 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9910 xid:136509 sent:136509 recv:137612 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_3 07:53:59.569 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=datanode-663-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1733:49999, readPos=1733:50000} to 1794:-1 since ledger consumed completely 07:53:59.569 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1733:49999, readPos=1733:50000} to 1794:-1 since ledger consumed completely 07:53:59.569 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-632-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1733:49999, readPos=1733:50000} to 1794:-1 since ledger consumed completely 07:53:59.569 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-633-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1733:49999, readPos=1733:50000} to 1794:-1 since ledger consumed completely 07:53:59.569 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-634-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1733:49999, readPos=1733:50000} to 1794:-1 since ledger consumed completely 07:53:59.569 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-636-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1733:49999, readPos=1733:50000} to 1794:-1 since ledger consumed completely 07:54:00.278 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_4] Closing ledger 1734 for being full 07:54:00.315 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Creating a new ledger 07:54:00.315 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 52], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:54:00.317 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1795 07:54:00.317 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Created new ledger 1795 07:54:00.322 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9920 xid:136510 sent:136510 recv:137614 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_4 07:54:00.322 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true, ackPos=1734:49999, readPos=1734:50000} to 1795:-1 since ledger consumed completely 07:54:00.322 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-632-by-dev-rootcoord-dml_4_450684035886621258v1-true, ackPos=1734:49999, readPos=1734:50000} to 1795:-1 since ledger consumed completely 07:54:00.322 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-633-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1734:49999, readPos=1734:50000} to 1795:-1 since ledger consumed completely 07:54:00.322 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-634-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1734:49999, readPos=1734:50000} to 1795:-1 since ledger consumed completely 07:54:00.322 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-636-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1734:49999, readPos=1734:50000} to 1795:-1 since ledger consumed completely 07:56:17.359 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_1] Closing ledger 1717 for being full 07:56:17.359 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_0] Closing ledger 1718 for being full 07:56:17.360 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_2] Closing ledger 1716 for being full 07:56:17.553 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_1] Creating a new ledger 07:56:17.554 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_1] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:56:17.554 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_0] Creating a new ledger 07:56:17.554 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_0] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 48], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:56:17.554 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_2] Creating a new ledger 07:56:17.554 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_2] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 50], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 07:56:17.556 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1796 07:56:17.556 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_1] Created new ledger 1796 07:56:17.556 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1797 07:56:17.556 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_0] Created new ledger 1797 07:56:17.556 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1798 07:56:17.556 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_2] Created new ledger 1798 07:56:17.557 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9925 xid:136586 sent:136586 recv:137691 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_1 07:56:17.557 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_1, name=datanode-662-by-dev-rootcoord-dml_1_450684035915234141v2-true, ackPos=1717:49999, readPos=1717:50000} to 1796:-1 since ledger consumed completely 07:56:17.557 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_1, name=datanode-663-by-dev-rootcoord-dml_1_450684035886220611v1-true, ackPos=1717:49999, readPos=1717:50000} to 1796:-1 since ledger consumed completely 07:56:17.557 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_1, name=querynode-633-by-dev-rootcoord-dml_1_450684035915234141v2-true, ackPos=1717:49999, readPos=1717:50000} to 1796:-1 since ledger consumed completely 07:56:17.557 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_1, name=querynode-634-by-dev-rootcoord-dml_1_450684035886220611v1-true, ackPos=1717:49999, readPos=1717:50000} to 1796:-1 since ledger consumed completely 07:56:17.557 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_1, name=querynode-636-by-dev-rootcoord-dml_1_450684035886220611v1-true, ackPos=1717:49999, readPos=1717:50000} to 1796:-1 since ledger consumed completely 07:56:17.557 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9925 xid:136586 sent:136586 recv:137693 queuedpkts:0 pendingresp:0 queuedevents:1] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_0 07:56:17.558 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9925 xid:136586 sent:136586 recv:137693 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_2 07:56:17.558 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_2, name=datanode-662-by-dev-rootcoord-dml_2_450684035915636293v0-true, ackPos=1716:49999, readPos=1716:50000} to 1798:-1 since ledger consumed completely 07:56:17.558 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_2, name=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true, ackPos=1716:49999, readPos=1716:50000} to 1798:-1 since ledger consumed completely 07:56:17.558 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_2, name=querynode-632-by-dev-rootcoord-dml_2_450684035886220611v2-true, ackPos=1716:49999, readPos=1716:50000} to 1798:-1 since ledger consumed completely 07:56:17.558 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_0, name=datanode-662-by-dev-rootcoord-dml_0_450684035886220611v0-true, ackPos=1718:49999, readPos=1718:50000} to 1797:-1 since ledger consumed completely 07:56:17.558 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_2, name=querynode-633-by-dev-rootcoord-dml_2_450684035915636293v0-true, ackPos=1716:49999, readPos=1716:50000} to 1798:-1 since ledger consumed completely 07:56:17.558 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_0, name=querynode-634-by-dev-rootcoord-dml_0_450684035915234141v1-true, ackPos=1718:49999, readPos=1718:50000} to 1797:-1 since ledger consumed completely 07:56:17.558 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_0, name=querynode-636-by-dev-rootcoord-dml_0_450684035915234141v1-true, ackPos=1718:49999, readPos=1718:50000} to 1797:-1 since ledger consumed completely 07:57:33.306 [db-storage-cleanup-10-1] INFO org.apache.bookkeeper.bookie.storage.ldb.EntryLocationIndex - Deleting indexes for ledgers: [1533, 1534, 1535] 07:57:35.538 [db-storage-cleanup-10-1] INFO org.apache.bookkeeper.bookie.storage.ldb.EntryLocationIndex - Deleted indexes for 150000 entries from 3 ledgers in 2.232 seconds 08:10:05.179 [BookieJournal-3181] INFO org.apache.bookkeeper.bookie.JournalChannel - Opening journal data/standalone/bookkeeper0/current/190621af979.txn 08:10:33.312 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.Journal - garbage collected journal 190621af973.txn 08:16:33.236 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true 08:16:33.236 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 361 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=361, logFile=data/standalone/bookkeeper0/current/169.log, ledgerIdAssigned=-1}]. 08:16:33.239 [pool-6-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/16b.log for logId 363. 08:16:33.384 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerForSingleEntryLog - Synced entry logger 361 to disk. 08:17:33.213 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Extracting entry log meta from entryLogId: 361 08:34:02.542 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-replicate-msg] Closing ledger 1712 for being full 08:34:02.554 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-replicate-msg] Creating a new ledger 08:34:02.554 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-replicate-msg] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 101, 112, 108, 105, 99, 97, 116, 101, 45, 109, 115, 103], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 08:34:02.558 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1799 08:34:02.558 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-replicate-msg] Created new ledger 1799 08:34:02.559 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9940 xid:137841 sent:137841 recv:138949 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-replicate-msg 08:39:33.239 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true 08:39:33.239 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 362 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=362, logFile=data/standalone/bookkeeper0/current/16a.log, ledgerIdAssigned=-1}]. 08:39:33.243 [pool-6-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/16c.log for logId 364. 08:39:33.441 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerForSingleEntryLog - Synced entry logger 362 to disk. 08:42:33.213 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Extracting entry log meta from entryLogId: 362 08:44:05.583 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_5] Closing ledger 1793 for being full 08:44:05.682 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Creating a new ledger 08:44:05.682 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 53], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 08:44:05.684 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1800 08:44:05.684 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Created new ledger 1800 08:44:05.685 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9945 xid:138176 sent:138176 recv:139285 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_5 08:44:05.685 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=datanode-662-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1793:49999, readPos=1793:50000} to 1800:-1 since ledger consumed completely 08:44:05.685 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true, ackPos=1793:49999, readPos=1793:50000} to 1800:-1 since ledger consumed completely 08:44:05.685 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-633-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1793:49999, readPos=1793:50000} to 1800:-1 since ledger consumed completely 08:44:05.685 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-634-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1793:49999, readPos=1793:50000} to 1800:-1 since ledger consumed completely 08:44:05.685 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-636-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1793:49999, readPos=1793:50000} to 1800:-1 since ledger consumed completely 08:44:06.148 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_3] Closing ledger 1794 for being full 08:44:06.155 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Creating a new ledger 08:44:06.155 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 51], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 08:44:06.157 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1801 08:44:06.157 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Created new ledger 1801 08:44:06.158 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9945 xid:138176 sent:138176 recv:139286 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_3 08:44:06.158 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=datanode-663-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1794:49999, readPos=1794:50000} to 1801:-1 since ledger consumed completely 08:44:06.158 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1794:49999, readPos=1794:50000} to 1801:-1 since ledger consumed completely 08:44:06.158 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-632-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1794:49999, readPos=1794:50000} to 1801:-1 since ledger consumed completely 08:44:06.158 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-633-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1794:49999, readPos=1794:50000} to 1801:-1 since ledger consumed completely 08:44:06.158 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-634-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1794:49999, readPos=1794:50000} to 1801:-1 since ledger consumed completely 08:44:06.158 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-636-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1794:49999, readPos=1794:50000} to 1801:-1 since ledger consumed completely 08:44:06.563 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_4] Closing ledger 1795 for being full 08:44:06.589 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Creating a new ledger 08:44:06.589 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 52], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 08:44:06.591 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1802 08:44:06.591 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Created new ledger 1802 08:44:06.592 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9945 xid:138176 sent:138176 recv:139287 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_4 08:44:06.592 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true, ackPos=1795:49999, readPos=1795:50000} to 1802:-1 since ledger consumed completely 08:44:06.592 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-632-by-dev-rootcoord-dml_4_450684035886621258v1-true, ackPos=1795:49999, readPos=1795:50000} to 1802:-1 since ledger consumed completely 08:44:06.592 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-633-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1795:49999, readPos=1795:50000} to 1802:-1 since ledger consumed completely 08:44:06.592 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-634-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1795:49999, readPos=1795:50000} to 1802:-1 since ledger consumed completely 08:44:06.592 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-636-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1795:49999, readPos=1795:50000} to 1802:-1 since ledger consumed completely 08:47:32.553 [PurgeTask] INFO org.apache.zookeeper.server.DatadirCleanupManager - Purge task started. 08:47:32.553 [PurgeTask] INFO org.apache.zookeeper.server.persistence.FileTxnSnapLog - zookeeper.snapshot.trust.empty : true 08:47:32.554 [PurgeTask] INFO org.apache.zookeeper.server.DatadirCleanupManager - Purge task completed. 08:47:33.215 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Enter minor compaction, suspendMinor false 08:47:33.215 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Do compaction to compact those files lower than 0.20000000298023224 08:51:04.356 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 249 after compaction 08:51:04.356 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 249 08:56:13.162 [BookieJournal-3181] INFO org.apache.bookkeeper.bookie.JournalChannel - Opening journal data/standalone/bookkeeper0/current/190621af97a.txn 08:56:33.420 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.Journal - garbage collected journal 190621af974.txn 08:56:56.810 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 233 after compaction 08:56:56.810 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 233 08:58:33.240 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true 08:58:33.240 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 363 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=363, logFile=data/standalone/bookkeeper0/current/16b.log, ledgerIdAssigned=-1}]. 08:58:33.243 [pool-6-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/16d.log for logId 365. 08:58:33.494 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerForSingleEntryLog - Synced entry logger 363 to disk. 09:01:33.937 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9960 xid:138761 sent:138761 recv:139873 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_5 09:01:33.937 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] End TrimConsumedLedgers. ledgers=14 totalSize=8597240943 09:01:33.937 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Removing ledger 1536 - size: 796303034 09:01:33.938 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9960 xid:138761 sent:138761 recv:139874 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_4 09:01:33.938 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9960 xid:138761 sent:138761 recv:139875 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_3 09:01:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] End TrimConsumedLedgers. ledgers=14 totalSize=8594004442 09:01:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Removing ledger 1538 - size: 796003163 09:01:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] End TrimConsumedLedgers. ledgers=14 totalSize=8594646035 09:01:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Removing ledger 1537 - size: 795637890 09:04:19.147 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 221 after compaction 09:04:19.147 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 221 09:04:54.000 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 324 after compaction 09:04:54.000 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 324 09:06:36.532 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 238 after compaction 09:06:36.532 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 238 09:07:13.861 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 322 after compaction 09:07:13.861 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 322 09:07:17.407 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true 09:07:17.408 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 364 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=364, logFile=data/standalone/bookkeeper0/current/16c.log, ledgerIdAssigned=-1}]. 09:07:17.410 [pool-6-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/16e.log for logId 366. 09:07:33.235 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerForSingleEntryLog - Synced entry logger 364 to disk. 09:09:21.098 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 229 after compaction 09:09:21.099 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 229 09:10:07.287 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 323 after compaction 09:10:07.287 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 323 09:12:13.091 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 224 after compaction 09:12:13.091 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 224 09:12:13.091 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Compaction: entry log usage buckets[10% 20% 30% 40% 50% 60% 70% 80% 90% 100%] = [1, 8, 31, 5, 2, 1, 5, 0, 1, 36], compacted [1, 8, 0, 0, 0, 0, 0, 0, 0, 0] 09:12:13.091 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Extracting entry log meta from entryLogId: 363 09:12:13.091 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Extracting entry log meta from entryLogId: 364 09:12:33.290 [db-storage-cleanup-10-1] INFO org.apache.bookkeeper.bookie.storage.ldb.EntryLocationIndex - Deleting indexes for ledgers: [1536, 1537, 1538] 09:12:38.253 [db-storage-cleanup-10-1] INFO org.apache.bookkeeper.bookie.storage.ldb.EntryLocationIndex - Deleted indexes for 150000 entries from 3 ledgers in 4.962 seconds 09:16:33.250 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true 09:16:33.251 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 365 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=365, logFile=data/standalone/bookkeeper0/current/16d.log, ledgerIdAssigned=-1}]. 09:16:33.255 [pool-6-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/16f.log for logId 367. 09:16:33.545 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerForSingleEntryLog - Synced entry logger 365 to disk. 09:17:33.213 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Extracting entry log meta from entryLogId: 365 09:26:49.555 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_14] Closing ledger 1728 for being full 09:26:49.555 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_13] Closing ledger 1730 for being full 09:26:49.555 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_9] Closing ledger 1722 for being full 09:26:49.555 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_7] Closing ledger 1723 for being full 09:26:49.555 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_10] Closing ledger 1726 for being full 09:26:49.555 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_12] Closing ledger 1727 for being full 09:26:49.555 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_11] Closing ledger 1724 for being full 09:26:49.555 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_8] Closing ledger 1731 for being full 09:26:49.555 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_15] Closing ledger 1725 for being full 09:26:49.555 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_6] Closing ledger 1729 for being full 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_10] Creating a new ledger 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_10] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 48], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_11] Creating a new ledger 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_11] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 49], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_13] Creating a new ledger 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_13] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 51], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_15] Creating a new ledger 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_12] Creating a new ledger 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_15] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 53], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_12] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 50], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_9] Creating a new ledger 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_6] Creating a new ledger 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_9] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 57], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_6] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 54], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_14] Creating a new ledger 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_14] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49, 52], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_7] Creating a new ledger 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_7] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 55], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_8] Creating a new ledger 09:26:49.754 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_8] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 56], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:26:49.756 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1803 09:26:49.756 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_10] Created new ledger 1803 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1804 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_11] Created new ledger 1804 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1805 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_13] Created new ledger 1805 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1806 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_15] Created new ledger 1806 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1807 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_12] Created new ledger 1807 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1808 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_9] Created new ledger 1808 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1809 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_6] Created new ledger 1809 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1810 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_14] Created new ledger 1810 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1811 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_7] Created new ledger 1811 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1812 09:26:49.757 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_8] Created new ledger 1812 09:26:49.758 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9976 xid:139600 sent:139600 recv:140715 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_10 09:26:49.758 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_10, name=datanode-664-by-dev-rootcoord-dml_10_450684035915233807v2-true, ackPos=1726:49999, readPos=1726:50000} to 1803:-1 since ledger consumed completely 09:26:49.758 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_10, name=querynode-632-by-dev-rootcoord-dml_10_450684035915233807v2-true, ackPos=1726:49999, readPos=1726:50000} to 1803:-1 since ledger consumed completely 09:26:49.758 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_10, name=querynode-633-by-dev-rootcoord-dml_10_450684035915233807v2-true, ackPos=1726:49999, readPos=1726:50000} to 1803:-1 since ledger consumed completely 09:26:49.759 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9976 xid:139600 sent:139600 recv:140716 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_11 09:26:49.759 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9976 xid:139600 sent:139600 recv:140719 queuedpkts:0 pendingresp:0 queuedevents:1] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_13 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_11, name=datanode-662-by-dev-rootcoord-dml_11_450684035915233981v0-true, ackPos=1724:49999, readPos=1724:50000} to 1804:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_13, name=datanode-663-by-dev-rootcoord-dml_13_450684035915233981v2-true, ackPos=1730:49999, readPos=1730:50000} to 1805:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_11, name=querynode-633-by-dev-rootcoord-dml_11_450684035915233981v0-true, ackPos=1724:49999, readPos=1724:50000} to 1804:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_13, name=querynode-632-by-dev-rootcoord-dml_13_450684035915233981v2-true, ackPos=1730:49999, readPos=1730:50000} to 1805:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_11, name=querynode-634-by-dev-rootcoord-dml_11_450684035915233981v0-true, ackPos=1724:49999, readPos=1724:50000} to 1804:-1 since ledger consumed completely 09:26:49.759 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9976 xid:139600 sent:139600 recv:140722 queuedpkts:0 pendingresp:0 queuedevents:4] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_15 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_14, name=datanode-663-by-dev-rootcoord-dml_14_450684035915234141v0-true, ackPos=1728:49999, readPos=1728:50000} to 1810:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_15, name=datanode-664-by-dev-rootcoord-dml_15_450684035915233807v0-true, ackPos=1725:49999, readPos=1725:50000} to 1806:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_13, name=querynode-633-by-dev-rootcoord-dml_13_450684035915233981v2-true, ackPos=1730:49999, readPos=1730:50000} to 1805:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_14, name=querynode-632-by-dev-rootcoord-dml_14_450684035915234141v0-true, ackPos=1728:49999, readPos=1728:50000} to 1810:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_6, name=datanode-663-by-dev-rootcoord-dml_6_450684035887223084v0-true, ackPos=1729:49999, readPos=1729:50000} to 1809:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_14, name=querynode-636-by-dev-rootcoord-dml_14_450684035915234141v0-true, ackPos=1728:49999, readPos=1728:50000} to 1810:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_6, name=querynode-632-by-dev-rootcoord-dml_6_450684035962993998v1-true, ackPos=1729:49999, readPos=1729:50000} to 1809:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_15, name=querynode-632-by-dev-rootcoord-dml_15_450684035915233807v0-true, ackPos=1725:49999, readPos=1725:50000} to 1806:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_12, name=datanode-662-by-dev-rootcoord-dml_12_450684035915233981v1-true, ackPos=1727:49999, readPos=1727:50000} to 1807:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_7, name=datanode-662-by-dev-rootcoord-dml_7_450684035887223084v1-true, ackPos=1723:49999, readPos=1723:50000} to 1811:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_7, name=datanode-663-by-dev-rootcoord-dml_7_450684035962993998v2-true, ackPos=1723:49999, readPos=1723:50000} to 1811:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_15, name=querynode-636-by-dev-rootcoord-dml_15_450684035915233807v0-true, ackPos=1725:49999, readPos=1725:50000} to 1806:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_7, name=querynode-632-by-dev-rootcoord-dml_7_450684035962993998v2-true, ackPos=1723:49999, readPos=1723:50000} to 1811:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_6, name=querynode-636-by-dev-rootcoord-dml_6_450684035962993998v1-true, ackPos=1729:49999, readPos=1729:50000} to 1809:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_7, name=querynode-633-by-dev-rootcoord-dml_7_450684035962993998v2-true, ackPos=1723:49999, readPos=1723:50000} to 1811:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_12, name=querynode-634-by-dev-rootcoord-dml_12_450684035915233981v1-true, ackPos=1727:49999, readPos=1727:50000} to 1807:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_8, name=datanode-664-by-dev-rootcoord-dml_8_450684035887223084v2-true, ackPos=1731:49999, readPos=1731:50000} to 1812:-1 since ledger consumed completely 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_12, name=querynode-636-by-dev-rootcoord-dml_12_450684035915233981v1-true, ackPos=1727:49999, readPos=1727:50000} to 1807:-1 since ledger consumed completely 09:26:49.759 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9976 xid:139600 sent:139600 recv:140724 queuedpkts:0 pendingresp:0 queuedevents:5] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_12 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_8, name=querynode-632-by-dev-rootcoord-dml_8_450684035887223084v2-true, ackPos=1731:49999, readPos=1731:50000} to 1812:-1 since ledger consumed completely 09:26:49.759 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9976 xid:139600 sent:139600 recv:140724 queuedpkts:0 pendingresp:0 queuedevents:4] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_9 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_9, name=datanode-663-by-dev-rootcoord-dml_9_450684035915233807v1-true, ackPos=1722:49999, readPos=1722:50000} to 1808:-1 since ledger consumed completely 09:26:49.759 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9976 xid:139600 sent:139600 recv:140724 queuedpkts:0 pendingresp:0 queuedevents:3] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_6 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_9, name=querynode-633-by-dev-rootcoord-dml_9_450684035915233807v1-true, ackPos=1722:49999, readPos=1722:50000} to 1808:-1 since ledger consumed completely 09:26:49.759 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9976 xid:139600 sent:139600 recv:140724 queuedpkts:0 pendingresp:0 queuedevents:2] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_14 09:26:49.759 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_9, name=querynode-634-by-dev-rootcoord-dml_9_450684035915233807v1-true, ackPos=1722:49999, readPos=1722:50000} to 1808:-1 since ledger consumed completely 09:26:49.759 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9976 xid:139600 sent:139600 recv:140724 queuedpkts:0 pendingresp:0 queuedevents:1] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_7 09:26:49.759 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:9976 xid:139600 sent:139600 recv:140724 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_8 09:29:10.844 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_5] Closing ledger 1800 for being full 09:29:10.846 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Creating a new ledger 09:29:10.846 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 53], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:29:10.848 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1813 09:29:10.848 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Created new ledger 1813 09:29:10.849 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10016 xid:139686 sent:139686 recv:140811 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_5 09:29:10.849 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=datanode-662-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1800:49999, readPos=1800:50000} to 1813:-1 since ledger consumed completely 09:29:10.850 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true, ackPos=1800:49999, readPos=1800:50000} to 1813:-1 since ledger consumed completely 09:29:10.850 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-633-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1800:49999, readPos=1800:50000} to 1813:-1 since ledger consumed completely 09:29:10.850 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-634-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1800:49999, readPos=1800:50000} to 1813:-1 since ledger consumed completely 09:29:10.850 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-636-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1800:49999, readPos=1800:50000} to 1813:-1 since ledger consumed completely 09:29:11.013 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_3] Closing ledger 1801 for being full 09:29:11.045 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Creating a new ledger 09:29:11.045 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 51], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:29:11.047 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1814 09:29:11.047 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Created new ledger 1814 09:29:11.053 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10016 xid:139686 sent:139686 recv:140812 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_3 09:29:11.053 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=datanode-663-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1801:49999, readPos=1801:50000} to 1814:-1 since ledger consumed completely 09:29:11.054 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1801:49999, readPos=1801:50000} to 1814:-1 since ledger consumed completely 09:29:11.054 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-632-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1801:49999, readPos=1801:50000} to 1814:-1 since ledger consumed completely 09:29:11.054 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-633-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1801:49999, readPos=1801:50000} to 1814:-1 since ledger consumed completely 09:29:11.054 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-634-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1801:49999, readPos=1801:50000} to 1814:-1 since ledger consumed completely 09:29:11.054 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-636-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1801:49999, readPos=1801:50000} to 1814:-1 since ledger consumed completely 09:29:11.424 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_4] Closing ledger 1802 for being full 09:29:11.430 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Creating a new ledger 09:29:11.430 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 52], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 09:29:11.433 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1815 09:29:11.433 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Created new ledger 1815 09:29:11.434 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10026 xid:139689 sent:139689 recv:140816 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_4 09:29:11.434 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true, ackPos=1802:49999, readPos=1802:50000} to 1815:-1 since ledger consumed completely 09:29:11.434 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-632-by-dev-rootcoord-dml_4_450684035886621258v1-true, ackPos=1802:49999, readPos=1802:50000} to 1815:-1 since ledger consumed completely 09:29:11.434 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-633-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1802:49999, readPos=1802:50000} to 1815:-1 since ledger consumed completely 09:29:11.434 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-634-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1802:49999, readPos=1802:50000} to 1815:-1 since ledger consumed completely 09:29:11.434 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-636-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1802:49999, readPos=1802:50000} to 1815:-1 since ledger consumed completely 09:36:05.666 [BookieJournal-3181] INFO org.apache.bookkeeper.bookie.JournalChannel - Opening journal data/standalone/bookkeeper0/current/190621af97b.txn 09:36:33.234 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true 09:36:33.234 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 366 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=366, logFile=data/standalone/bookkeeper0/current/16e.log, ledgerIdAssigned=-1}]. 09:36:33.237 [pool-6-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/170.log for logId 368. 09:36:33.394 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerForSingleEntryLog - Synced entry logger 366 to disk. 09:36:33.404 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.Journal - garbage collected journal 190621af975.txn 09:37:33.213 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Extracting entry log meta from entryLogId: 366 09:47:32.553 [PurgeTask] INFO org.apache.zookeeper.server.DatadirCleanupManager - Purge task started. 09:47:32.553 [PurgeTask] INFO org.apache.zookeeper.server.persistence.FileTxnSnapLog - zookeeper.snapshot.trust.empty : true 09:47:32.605 [PurgeTask] INFO org.apache.zookeeper.server.DatadirCleanupManager - Purge task completed. 09:55:33.237 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true 09:55:33.237 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 367 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=367, logFile=data/standalone/bookkeeper0/current/16f.log, ledgerIdAssigned=-1}]. 09:55:33.241 [pool-6-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/171.log for logId 369. 09:55:33.425 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerForSingleEntryLog - Synced entry logger 367 to disk. 09:55:33.937 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10031 xid:140566 sent:140566 recv:141695 queuedpkts:0 pendingresp:0 queuedevents:1] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_5 09:55:33.937 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] End TrimConsumedLedgers. ledgers=14 totalSize=8613775988 09:55:33.937 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Removing ledger 1542 - size: 779458590 09:55:33.937 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] End TrimConsumedLedgers. ledgers=14 totalSize=8611666195 09:55:33.937 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Removing ledger 1544 - size: 778573079 09:55:33.937 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10031 xid:140566 sent:140566 recv:141695 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_4 09:55:33.938 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10031 xid:140566 sent:140566 recv:141696 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_3 09:55:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] End TrimConsumedLedgers. ledgers=14 totalSize=8612006002 09:55:33.938 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Removing ledger 1543 - size: 778815063 09:57:33.213 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Extracting entry log meta from entryLogId: 367 09:57:33.427 [db-storage-cleanup-10-1] INFO org.apache.bookkeeper.bookie.storage.ldb.EntryLocationIndex - Deleting indexes for ledgers: [1542, 1543, 1544] 09:57:35.414 [db-storage-cleanup-10-1] INFO org.apache.bookkeeper.bookie.storage.ldb.EntryLocationIndex - Deleted indexes for 150000 entries from 3 ledgers in 1.987 seconds 10:07:22.152 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_1] Closing ledger 1796 for being full 10:07:22.152 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_2] Closing ledger 1798 for being full 10:07:22.152 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_0] Closing ledger 1797 for being full 10:07:22.352 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_0] Creating a new ledger 10:07:22.352 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_0] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 48], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 10:07:22.352 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_2] Creating a new ledger 10:07:22.352 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_2] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 50], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 10:07:22.352 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_1] Creating a new ledger 10:07:22.352 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_1] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 49], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 10:07:22.354 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1816 10:07:22.354 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_0] Created new ledger 1816 10:07:22.355 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1817 10:07:22.355 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_2] Created new ledger 1817 10:07:22.355 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1818 10:07:22.355 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_1] Created new ledger 1818 10:07:22.356 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10037 xid:140958 sent:140958 recv:142089 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_0 10:07:22.356 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_0, name=datanode-662-by-dev-rootcoord-dml_0_450684035886220611v0-true, ackPos=1797:49999, readPos=1797:50000} to 1816:-1 since ledger consumed completely 10:07:22.356 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_0, name=querynode-634-by-dev-rootcoord-dml_0_450684035915234141v1-true, ackPos=1797:49999, readPos=1797:50000} to 1816:-1 since ledger consumed completely 10:07:22.356 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_0, name=querynode-636-by-dev-rootcoord-dml_0_450684035915234141v1-true, ackPos=1797:49999, readPos=1797:50000} to 1816:-1 since ledger consumed completely 10:07:22.356 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10037 xid:140958 sent:140958 recv:142091 queuedpkts:0 pendingresp:0 queuedevents:1] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_2 10:07:22.356 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10037 xid:140958 sent:140958 recv:142091 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_1 10:07:22.356 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_1, name=datanode-662-by-dev-rootcoord-dml_1_450684035915234141v2-true, ackPos=1796:49999, readPos=1796:50000} to 1818:-1 since ledger consumed completely 10:07:22.356 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_2, name=datanode-662-by-dev-rootcoord-dml_2_450684035915636293v0-true, ackPos=1798:49999, readPos=1798:50000} to 1817:-1 since ledger consumed completely 10:07:22.356 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_1, name=datanode-663-by-dev-rootcoord-dml_1_450684035886220611v1-true, ackPos=1796:49999, readPos=1796:50000} to 1818:-1 since ledger consumed completely 10:07:22.356 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_1, name=querynode-633-by-dev-rootcoord-dml_1_450684035915234141v2-true, ackPos=1796:49999, readPos=1796:50000} to 1818:-1 since ledger consumed completely 10:07:22.356 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_2, name=datanode-664-by-dev-rootcoord-dml_2_450684035886220611v2-true, ackPos=1798:49999, readPos=1798:50000} to 1817:-1 since ledger consumed completely 10:07:22.356 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_1, name=querynode-634-by-dev-rootcoord-dml_1_450684035886220611v1-true, ackPos=1796:49999, readPos=1796:50000} to 1818:-1 since ledger consumed completely 10:07:22.357 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_1, name=querynode-636-by-dev-rootcoord-dml_1_450684035886220611v1-true, ackPos=1796:49999, readPos=1796:50000} to 1818:-1 since ledger consumed completely 10:07:22.357 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_2, name=querynode-632-by-dev-rootcoord-dml_2_450684035886220611v2-true, ackPos=1798:49999, readPos=1798:50000} to 1817:-1 since ledger consumed completely 10:07:22.357 [bookkeeper-ml-scheduler-OrderedScheduler-2-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_2, name=querynode-633-by-dev-rootcoord-dml_2_450684035915636293v0-true, ackPos=1798:49999, readPos=1798:50000} to 1817:-1 since ledger consumed completely 10:12:00.468 [BookKeeperClientWorker-OrderedExecutor-1-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_5] Closing ledger 1813 for being full 10:12:00.469 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Creating a new ledger 10:12:00.469 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 53], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 10:12:00.471 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1819 10:12:00.471 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_5] Created new ledger 1819 10:12:00.472 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10052 xid:141113 sent:141113 recv:142247 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_5 10:12:00.472 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=datanode-662-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1813:49999, readPos=1813:50000} to 1819:-1 since ledger consumed completely 10:12:00.472 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=datanode-664-by-dev-rootcoord-dml_5_450684035886621258v2-true, ackPos=1813:49999, readPos=1813:50000} to 1819:-1 since ledger consumed completely 10:12:00.472 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-633-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1813:49999, readPos=1813:50000} to 1819:-1 since ledger consumed completely 10:12:00.472 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-634-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1813:49999, readPos=1813:50000} to 1819:-1 since ledger consumed completely 10:12:00.472 [bookkeeper-ml-scheduler-OrderedScheduler-1-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_5, name=querynode-636-by-dev-rootcoord-dml_5_450684035962993998v0-true, ackPos=1813:49999, readPos=1813:50000} to 1819:-1 since ledger consumed completely 10:12:00.880 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_3] Closing ledger 1814 for being full 10:12:00.918 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Creating a new ledger 10:12:00.918 [BookKeeperClientWorker-OrderedExecutor-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 51], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 10:12:00.920 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1820 10:12:00.920 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_3] Created new ledger 1820 10:12:00.921 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10052 xid:141113 sent:141113 recv:142248 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_3 10:12:00.921 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=datanode-663-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1814:49999, readPos=1814:50000} to 1820:-1 since ledger consumed completely 10:12:00.921 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=datanode-664-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1814:49999, readPos=1814:50000} to 1820:-1 since ledger consumed completely 10:12:00.922 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-632-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1814:49999, readPos=1814:50000} to 1820:-1 since ledger consumed completely 10:12:00.922 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-633-by-dev-rootcoord-dml_3_450684035886621258v0-true, ackPos=1814:49999, readPos=1814:50000} to 1820:-1 since ledger consumed completely 10:12:00.922 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-634-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1814:49999, readPos=1814:50000} to 1820:-1 since ledger consumed completely 10:12:00.922 [bookkeeper-ml-scheduler-OrderedScheduler-3-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_3, name=querynode-636-by-dev-rootcoord-dml_3_450684035915636293v1-true, ackPos=1814:49999, readPos=1814:50000} to 1820:-1 since ledger consumed completely 10:12:02.139 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.OpAddEntry - [public/default/persistent/by-dev-rootcoord-dml_4] Closing ledger 1815 for being full 10:12:02.141 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Creating a new ledger 10:12:02.141 [BookKeeperClientWorker-OrderedExecutor-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Creating ledger, metadata: {component=[109, 97, 110, 97, 103, 101, 100, 45, 108, 101, 100, 103, 101, 114], pulsar/managed-ledger=[112, 117, 98, 108, 105, 99, 47, 100, 101, 102, 97, 117, 108, 116, 47, 112, 101, 114, 115, 105, 115, 116, 101, 110, 116, 47, 98, 121, 45, 100, 101, 118, 45, 114, 111, 111, 116, 99, 111, 111, 114, 100, 45, 100, 109, 108, 95, 52], application=[112, 117, 108, 115, 97, 114]} - metadata ops timeout : 60 seconds 10:12:02.144 [main-EventThread] INFO org.apache.bookkeeper.client.LedgerCreateOp - Ensemble: [127.0.0.1:3181] for ledger: 1821 10:12:02.144 [main-EventThread] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - [public/default/persistent/by-dev-rootcoord-dml_4] Created new ledger 1821 10:12:02.145 [pulsar-ordered-OrderedExecutor-2-0-EventThread] INFO org.apache.pulsar.zookeeper.ZooKeeperCache - [State:CONNECTED Timeout:30000 sessionid:0x1027801c3b70004 local:/127.0.0.1:48912 remoteserver:localhost/127.0.0.1:2181 lastZxid:10062 xid:141114 sent:141114 recv:142250 queuedpkts:0 pendingresp:0 queuedevents:0] Received ZooKeeper watch event: WatchedEvent state:SyncConnected type:NodeDataChanged path:/managed-ledgers/public/default/persistent/by-dev-rootcoord-dml_4 10:12:02.145 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=datanode-664-by-dev-rootcoord-dml_4_450684035886621258v1-true, ackPos=1815:49999, readPos=1815:50000} to 1821:-1 since ledger consumed completely 10:12:02.145 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-632-by-dev-rootcoord-dml_4_450684035886621258v1-true, ackPos=1815:49999, readPos=1815:50000} to 1821:-1 since ledger consumed completely 10:12:02.145 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-633-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1815:49999, readPos=1815:50000} to 1821:-1 since ledger consumed completely 10:12:02.145 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-634-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1815:49999, readPos=1815:50000} to 1821:-1 since ledger consumed completely 10:12:02.145 [bookkeeper-ml-scheduler-OrderedScheduler-0-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Reset cursor:ManagedCursorImpl{ledger=public/default/persistent/by-dev-rootcoord-dml_4, name=querynode-636-by-dev-rootcoord-dml_4_450684035915636293v2-true, ackPos=1815:49999, readPos=1815:50000} to 1821:-1 since ledger consumed completely 10:12:33.215 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Enter minor compaction, suspendMinor false 10:12:33.215 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Do compaction to compact those files lower than 0.20000000298023224 10:13:06.973 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true 10:13:06.973 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 368 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=368, logFile=data/standalone/bookkeeper0/current/170.log, ledgerIdAssigned=-1}]. 10:13:06.976 [pool-6-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/172.log for logId 370. 10:13:13.316 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerForSingleEntryLog - Synced entry logger 368 to disk. 10:13:13.499 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 330 after compaction 10:13:13.499 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 330 10:13:19.078 [BookieJournal-3181] INFO org.apache.bookkeeper.bookie.JournalChannel - Opening journal data/standalone/bookkeeper0/current/190621af97c.txn 10:13:33.393 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.Journal - garbage collected journal 190621af976.txn 10:13:54.864 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 326 after compaction 10:13:54.865 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 326 10:14:40.524 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.EntryLogCompactor - Removing entry log 325 after compaction 10:14:40.524 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Removing entry log metadata for 325 10:14:40.524 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Compaction: entry log usage buckets[10% 20% 30% 40% 50% 60% 70% 80% 90% 100%] = [0, 3, 33, 5, 2, 2, 5, 0, 1, 35], compacted [0, 3, 0, 0, 0, 0, 0, 0, 0, 0] 10:17:33.213 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Extracting entry log meta from entryLogId: 368 10:21:33.245 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Creating a new entry log file : createNewLog = false, reachEntryLogLimit = true 10:21:33.245 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerBase - Flushing entry logger 369 back to filesystem, pending for syncing entry loggers : [BufferedChannel{logId=369, logFile=data/standalone/bookkeeper0/current/171.log, ledgerIdAssigned=-1}]. 10:21:33.248 [pool-6-thread-1] INFO org.apache.bookkeeper.bookie.EntryLoggerAllocator - Created new entry log file data/standalone/bookkeeper0/current/173.log for logId 371. 10:21:33.400 [SyncThread-7-1] INFO org.apache.bookkeeper.bookie.EntryLogManagerForSingleEntryLog - Synced entry logger 369 to disk. 10:22:33.213 [GarbageCollectorThread-11-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Extracting entry log meta from entryLogId: 369

stale[bot] commented 3 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.