Closed ufuk23 closed 2 years ago
is it three hundred or 300000? can we check the proxy log and see what's the detailed error?
it is 300.000, or 50.000, it makes no difference. How could I see my proxy log? The docker container is down when I get this error.
docker logs [OPTIONS] CONTAINER is the way you can get the log, or you can redirect log of milvus to a file by change log.file.rootPath
we will need to see at least the error message before we can judge
Here it is:
{"log":"\u0009/usr/local/go/src/bufio/bufio.go:101 +0x105\n","stream":"stderr","time":"2022-02-09T11:48:59.319096561Z"} {"log":"bufio.(Reader).Peek(0xc0040802a0, 0x1, 0x2, 0x0, 0x0, 0x0, 0xc00495e780)\n","stream":"stderr","time":"2022-02-09T11:48:59.319119782Z"} {"log":"\u0009/usr/local/go/src/bufio/bufio.go:139 +0x4f\n","stream":"stderr","time":"2022-02-09T11:48:59.31912882Z"} {"log":"net/http.(persistConn).readLoop(0xc00638e240)\n","stream":"stderr","time":"2022-02-09T11:48:59.319136789Z"} {"log":"\u0009/usr/local/go/src/net/http/transport.go:2040 +0x1a8\n","stream":"stderr","time":"2022-02-09T11:48:59.319160068Z"} {"log":"created by net/http.(Transport).dialConn\n","stream":"stderr","time":"2022-02-09T11:48:59.319169139Z"} {"log":"\u0009/usr/local/go/src/net/http/transport.go:1708 +0xcb7\n","stream":"stderr","time":"2022-02-09T11:48:59.319177175Z"} {"log":"\n","stream":"stderr","time":"2022-02-09T11:48:59.319184941Z"} {"log":"goroutine 442545 [select]:\n","stream":"stderr","time":"2022-02-09T11:48:59.319216528Z"} {"log":"net/http.(persistConn).writeLoop(0xc00227afc0)\n","stream":"stderr","time":"2022-02-09T11:48:59.319225542Z"} {"log":"\u0009/usr/local/go/src/net/http/transport.go:2340 +0x11c\n","stream":"stderr","time":"2022-02-09T11:48:59.31923339Z"} {"log":"created by net/http.(*Transport).dialConn\n","stream":"stderr","time":"2022-02-09T11:48:59.319566785Z"} {"log":"\u0009/usr/local/go/src/net/http/transport.go:1709 +0xcdc\n","stream":"stderr","time":"2022-02-09T11:48:59.319575161Z"}
And here is the python example console log:
RPC error: _execute_search_requests{'RPC start': '2022-02-09 16:28:35.416039', 'RPC error': '2022-02-09 16:28:37.828593'}
RPC error: search{'RPC start': '2022-02-09 16:28:35.393501', 'RPC error': '2022-02-09 16:28:37.828593'}
Traceback (most recent call last):
File "C:\Users\Aynur\PycharmProjects\featureExtraction\milvus2_search_test.py", line 96, in
Here it is:
{"log":"\u0009/usr/local/go/src/bufio/bufio.go:101 +0x105\n","stream":"stderr","time":"2022-02-09T11:48:59.319096561Z"} {"log":"bufio.(Reader).Peek(0xc0040802a0, 0x1, 0x2, 0x0, 0x0, 0x0, 0xc00495e780)\n","stream":"stderr","time":"2022-02-09T11:48:59.319119782Z"} {"log":"\u0009/usr/local/go/src/bufio/bufio.go:139 +0x4f\n","stream":"stderr","time":"2022-02-09T11:48:59.31912882Z"} {"log":"net/http.(persistConn).readLoop(0xc00638e240)\n","stream":"stderr","time":"2022-02-09T11:48:59.319136789Z"} {"log":"\u0009/usr/local/go/src/net/http/transport.go:2040 +0x1a8\n","stream":"stderr","time":"2022-02-09T11:48:59.319160068Z"} {"log":"created by net/http.(Transport).dialConn\n","stream":"stderr","time":"2022-02-09T11:48:59.319169139Z"} {"log":"\u0009/usr/local/go/src/net/http/transport.go:1708 +0xcb7\n","stream":"stderr","time":"2022-02-09T11:48:59.319177175Z"} {"log":"\n","stream":"stderr","time":"2022-02-09T11:48:59.319184941Z"} {"log":"goroutine 442545 [select]:\n","stream":"stderr","time":"2022-02-09T11:48:59.319216528Z"} {"log":"net/http.(persistConn).writeLoop(0xc00227afc0)\n","stream":"stderr","time":"2022-02-09T11:48:59.319225542Z"} {"log":"\u0009/usr/local/go/src/net/http/transport.go:2340 +0x11c\n","stream":"stderr","time":"2022-02-09T11:48:59.31923339Z"} {"log":"created by net/http.(*Transport).dialConn\n","stream":"stderr","time":"2022-02-09T11:48:59.319566785Z"} {"log":"\u0009/usr/local/go/src/net/http/transport.go:1709 +0xcdc\n","stream":"stderr","time":"2022-02-09T11:48:59.319575161Z"}
And here is the python example console log:
RPC error: _execute_search_requests{'RPC start': '2022-02-09 16:28:35.416039', 'RPC error': '2022-02-09 16:28:37.828593'} RPC error: search{'RPC start': '2022-02-09 16:28:35.393501', 'RPC error': '2022-02-09 16:28:37.828593'} Traceback (most recent call last): File "C:\Users\Aynur\PycharmProjects\featureExtraction\milvus2_search_test.py", line 96, in result = art.search(vec, "embeddings", search_params, limit=10, expr=express, output_fields=["artifact_type"]) File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\orm\collection.py", line 697, in search res = conn.search(self._name, data, anns_field, param, limit, expr, File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\decorators.py", line 47, in handler raise e File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\decorators.py", line 41, in handler return func(self, *args, kwargs) File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\decorators.py", line 82, in handler raise e File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\decorators.py", line 70, in handler return func(*args, *kwargs) File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\decorators.py", line 21, in handler return func(self, args, kwargs) File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\client\grpc_handler.py", line 444, in search return self._execute_search_requests(requests, timeout, *_kwargs) File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\decorators.py", line 82, in handler raise e File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\decorators.py", line 70, in handler return func(args, **kwargs) File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\client\grpc_handler.py", line 391, in _execute_search_requests raise pre_err File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\client\grpc_handler.py", line 379, in _execute_search_requests response = ft.result() File "C:\Users\Aynur\PycharmProjects\featureExtraction\venv\lib\site-packages\grpc_channel.py", line 744, in result raise self grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with: status = StatusCode.UNKNOWN details = "Stream removed" debug_error_string = "{"created":"@1644413317.833000000","description":"Error received from peer ipv4:10.1.37.185:19530","file":"src/core/lib/surface/call.cc","file_line":1068,"grpc_message":"Stream removed","grpc_status":2}"
This is the log from client side, do we have the log at server side, especially for the proxy? see if you don't know how to collect logs~ https://milvus.io/docs/v2.0.0/operational_faq.md I'm not sure if we can work on so many in expressions but is there a reason for using so much in?
I realised that milvus-standalone has many error logs, even has no collection. I used the docker-compose as default but don't know why I have those errors. I have the attu GUI by the way. Even if I remove the milvus containers and volumes and run docker compose up, it still has the same errors.
` [2022/02/09 17:50:15.555 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.555 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_39_431042895297118209v1] [SegmentID=431042922927095809] [NumOfRows=57]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431042922927095809] ["collection ID"=431042895297118209] ["partition ID"=431042895297118210] ["channel name"=by-dev-rootcoord-dml_39_431042895297118209v1]
[2022/02/09 17:50:15.555 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.554 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.555 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_35_431029268029636609v1] [SegmentID=431029268684996610] [NumOfRows=57]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431029268684996610] ["collection ID"=431029268029636609] ["partition ID"=431029268029636610] ["channel name"=by-dev-rootcoord-dml_35_431029268029636609v1]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.556 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_33_431029110114091009v1] [SegmentID=431029110769451009] [NumOfRows=57]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431029110769451009] ["collection ID"=431029110114091009] ["partition ID"=431029110114091010] ["channel name"=by-dev-rootcoord-dml_33_431029110114091009v1]
[2022/02/09 17:50:15.556 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.555 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431074894069104641"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.554 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.555 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_10_431028181783543809v0] [SegmentID=431028181822865409] [NumOfRows=43]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431028181822865409] ["collection ID"=431028181783543809] ["partition ID"=431028181783543810] ["channel name"=by-dev-rootcoord-dml_10_431028181783543809v0]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.555 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.557 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_1_430937036841549825v1] [SegmentID=430937036933300226] [NumOfRows=1497]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.557 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_12_431028197852971009v0] [SegmentID=431028198495223809] [NumOfRows=43]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431028198495223809] ["collection ID"=431028197852971009] ["partition ID"=431028197852971010] ["channel name"=by-dev-rootcoord-dml_12_431028197852971009v0]
[2022/02/09 17:50:15.557 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=430937036933300226] ["collection ID"=430937036841549825] ["partition ID"=430937036841549826] ["channel name"=by-dev-rootcoord-dml_1_430937036841549825v1]
[2022/02/09 17:50:15.557 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.557 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431042895297118209"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.556 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.557 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 430938917816500225"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.558 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_22_431028383581995009v0] [SegmentID=431028384211402753] [NumOfRows=43]
[2022/02/09 17:50:15.558 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431028384211402753] ["collection ID"=431028383581995009] ["partition ID"=431028383581995010] ["channel name"=by-dev-rootcoord-dml_22_431028383581995009v0]
[2022/02/09 17:50:15.558 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.558 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_31_431029027971268609v1] [SegmentID=431029028613521409] [NumOfRows=57]
[2022/02/09 17:50:15.558 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431029028613521409] ["collection ID"=431029027971268609] ["partition ID"=431029027971268610] ["channel name"=by-dev-rootcoord-dml_31_431029027971268609v1]
[2022/02/09 17:50:15.558 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.558 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.558 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431074894069104641 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431074894069104641"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.559 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431028197852971009"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.559 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 430937036841549825"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.559 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_7_431074894069104641v1] [error="describe collection 431074894069104641 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431074894069104641"]
[2022/02/09 17:50:15.559 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.559 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.559 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.559 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.559 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431028383581995009"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.559 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431076965393629185"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.557 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431029110114091009"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.559 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 430937036841549825 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937036841549825"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.560 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_1_430937036841549825v1] [error="describe collection 430937036841549825 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937036841549825"]
[2022/02/09 17:50:15.560 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_1_430937036841549825v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_1_430937036841549825v1, err: describe collection 430937036841549825 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937036841549825"]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431028197852971009"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 430938917816500225 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430938917816500225"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.558 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431029110114091009"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431029268029636609"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.555 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 430937249230618625 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937249230618625"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_13_431028197852971009v1] [error="describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_5_430937249230618625v1] [error="describe collection 430937249230618625 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937249230618625"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_13_431028197852971009v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_13_431028197852971009v1, err: describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_5_430937249230618625v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_5_430937249230618625v1, err: describe collection 430937249230618625 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937249230618625"]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431029027971268609"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_10_430938917816500225v0] [error="describe collection 430938917816500225 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430938917816500225"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_10_430938917816500225v0] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_10_430938917816500225v0, err: describe collection 430938917816500225 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430938917816500225"]
[2022/02/09 17:50:15.561 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431028383581995009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028383581995009"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_33_431029110114091009v1] [error="describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"]
[2022/02/09 17:50:15.561 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431076965393629185 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431076965393629185"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_33_431029110114091009v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_33_431029110114091009v1, err: describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_1_431076965393629185v1] [error="describe collection 431076965393629185 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431076965393629185"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_22_431028383581995009v0] [error="describe collection 431028383581995009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028383581995009"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_22_431028383581995009v0] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_22_431028383581995009v0, err: describe collection 431028383581995009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028383581995009"]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431042895297118209 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431042895297118209"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431028181783543809"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.562 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.559 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_7_431074894069104641v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_7_431074894069104641v1, err: describe collection 431074894069104641 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431074894069104641"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_12_431028197852971009v0] [error="describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_12_431028197852971009v0] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_12_431028197852971009v0, err: describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_1_431076965393629185v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_1_431076965393629185v1, err: describe collection 431076965393629185 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431076965393629185"]
[2022/02/09 17:50:15.562 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_39_431042895297118209v1] [error="describe collection 431042895297118209 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431042895297118209"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_39_431042895297118209v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_39_431042895297118209v1, err: describe collection 431042895297118209 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431042895297118209"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_32_431029110114091009v0] [error="describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_32_431029110114091009v0] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_32_431029110114091009v0, err: describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"]
[2022/02/09 17:50:15.563 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431029027971268609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029027971268609"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_31_431029027971268609v1] [error="describe collection 431029027971268609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029027971268609"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_31_431029027971268609v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_31_431029027971268609v1, err: describe collection 431029027971268609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029027971268609"]
[2022/02/09 17:50:15.563 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431028181783543809 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028181783543809"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_10_431028181783543809v0] [error="describe collection 431028181783543809 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028181783543809"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_10_431028181783543809v0] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_10_431028181783543809v0, err: describe collection 431028181783543809 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028181783543809"]
[2022/02/09 17:50:15.563 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431029268029636609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029268029636609"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_35_431029268029636609v1] [error="describe collection 431029268029636609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029268029636609"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_35_431029268029636609v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_35_431029268029636609v1, err: describe collection 431029268029636609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029268029636609"] `
I realised that milvus-standalone has many error logs, even has no collection. I used the docker-compose as default but don't know why I have those errors. I have the attu GUI by the way. Even if I remove the milvus containers and volumes and run docker compose up, it still has the same errors.
` [2022/02/09 17:50:15.555 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.555 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_39_431042895297118209v1] [SegmentID=431042922927095809] [NumOfRows=57]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431042922927095809] ["collection ID"=431042895297118209] ["partition ID"=431042895297118210] ["channel name"=by-dev-rootcoord-dml_39_431042895297118209v1]
[2022/02/09 17:50:15.555 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.554 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.555 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_35_431029268029636609v1] [SegmentID=431029268684996610] [NumOfRows=57]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431029268684996610] ["collection ID"=431029268029636609] ["partition ID"=431029268029636610] ["channel name"=by-dev-rootcoord-dml_35_431029268029636609v1]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.556 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_33_431029110114091009v1] [SegmentID=431029110769451009] [NumOfRows=57]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431029110769451009] ["collection ID"=431029110114091009] ["partition ID"=431029110114091010] ["channel name"=by-dev-rootcoord-dml_33_431029110114091009v1]
[2022/02/09 17:50:15.556 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.555 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431074894069104641"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.554 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.555 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_10_431028181783543809v0] [SegmentID=431028181822865409] [NumOfRows=43]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431028181822865409] ["collection ID"=431028181783543809] ["partition ID"=431028181783543810] ["channel name"=by-dev-rootcoord-dml_10_431028181783543809v0]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.555 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.557 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_1_430937036841549825v1] [SegmentID=430937036933300226] [NumOfRows=1497]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.557 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_12_431028197852971009v0] [SegmentID=431028198495223809] [NumOfRows=43]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431028198495223809] ["collection ID"=431028197852971009] ["partition ID"=431028197852971010] ["channel name"=by-dev-rootcoord-dml_12_431028197852971009v0]
[2022/02/09 17:50:15.557 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=430937036933300226] ["collection ID"=430937036841549825] ["partition ID"=430937036841549826] ["channel name"=by-dev-rootcoord-dml_1_430937036841549825v1]
[2022/02/09 17:50:15.557 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.557 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431042895297118209"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.556 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.555 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.556 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [minio_kv.go:92] ["MinioKV new MinioKV success."]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.557 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.557 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 430938917816500225"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.558 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_22_431028383581995009v0] [SegmentID=431028384211402753] [NumOfRows=43]
[2022/02/09 17:50:15.558 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431028384211402753] ["collection ID"=431028383581995009] ["partition ID"=431028383581995010] ["channel name"=by-dev-rootcoord-dml_22_431028383581995009v0]
[2022/02/09 17:50:15.558 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.558 +00:00] [INFO] [data_sync_service.go:186] ["Recover Segment NumOfRows form checkpoints"] [InsertChannel=by-dev-rootcoord-dml_31_431029027971268609v1] [SegmentID=431029028613521409] [NumOfRows=57]
[2022/02/09 17:50:15.558 +00:00] [DEBUG] [segment_replica.go:378] ["Add Flushed segment"] ["segment ID"=431029028613521409] ["collection ID"=431029027971268609] ["partition ID"=431029027971268610] ["channel name"=by-dev-rootcoord-dml_31_431029027971268609v1]
[2022/02/09 17:50:15.558 +00:00] [INFO] [segment_replica.go:415] ["statsBinlogs is empty"]
[2022/02/09 17:50:15.558 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.558 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431074894069104641 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431074894069104641"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.559 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431028197852971009"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.559 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 430937036841549825"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.559 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_7_431074894069104641v1] [error="describe collection 431074894069104641 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431074894069104641"]
[2022/02/09 17:50:15.559 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.559 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.559 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.559 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.559 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431028383581995009"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.559 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431076965393629185"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.557 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431029110114091009"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.559 +00:00] [DEBUG] [root_coord.go:1377] [DescribeCollection] [role=rootcoord] ["collection name"=] [msgID=0]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 430937036841549825 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937036841549825"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.560 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_1_430937036841549825v1] [error="describe collection 430937036841549825 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937036841549825"]
[2022/02/09 17:50:15.560 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_1_430937036841549825v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_1_430937036841549825v1, err: describe collection 430937036841549825 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937036841549825"]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431028197852971009"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 430938917816500225 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430938917816500225"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.558 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431029110114091009"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431029268029636609"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.555 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 430937249230618625 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937249230618625"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_13_431028197852971009v1] [error="describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_5_430937249230618625v1] [error="describe collection 430937249230618625 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937249230618625"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_13_431028197852971009v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_13_431028197852971009v1, err: describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_5_430937249230618625v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_5_430937249230618625v1, err: describe collection 430937249230618625 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430937249230618625"]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431029027971268609"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_10_430938917816500225v0] [error="describe collection 430938917816500225 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430938917816500225"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_10_430938917816500225v0] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_10_430938917816500225v0, err: describe collection 430938917816500225 from rootcoord wrong: DescribeCollection failed: can't find collection id : 430938917816500225"]
[2022/02/09 17:50:15.561 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431028383581995009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028383581995009"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_33_431029110114091009v1] [error="describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"]
[2022/02/09 17:50:15.561 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431076965393629185 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431076965393629185"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_33_431029110114091009v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_33_431029110114091009v1, err: describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_1_431076965393629185v1] [error="describe collection 431076965393629185 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431076965393629185"]
[2022/02/09 17:50:15.561 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_22_431028383581995009v0] [error="describe collection 431028383581995009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028383581995009"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_22_431028383581995009v0] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_22_431028383581995009v0, err: describe collection 431028383581995009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028383581995009"]
[2022/02/09 17:50:15.560 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431042895297118209 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431042895297118209"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.561 +00:00] [ERROR] [root_coord.go:1389] ["DescribeCollection failed"] [role=rootcoord] ["collection name"=] [msgID=0] [error="can't find collection id : 431028181783543809"] [stack="github.com/milvus-io/milvus/internal/rootcoord.(Core).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/rootcoord/root_coord.go:1389\ngithub.com/milvus-io/milvus/internal/distributed/rootcoord.(Server).DescribeCollection\n\t/go/src/github.com/milvus-io/milvus/internal/distributed/rootcoord/service.go:354\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler.func1\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:915\ngithub.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1\n\t/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38\ngithub.com/milvus-io/milvus/internal/proto/rootcoordpb._RootCoord_DescribeCollection_Handler\n\t/go/src/github.com/milvus-io/milvus/internal/proto/rootcoordpb/root_coord.pb.go:917\ngoogle.golang.org/grpc.(Server).processUnaryRPC\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1286\ngoogle.golang.org/grpc.(Server).handleStream\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:1609\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.2\n\t/go/pkg/mod/google.golang.org/grpc@v1.38.0/server.go:934"]
[2022/02/09 17:50:15.562 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.559 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_7_431074894069104641v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_7_431074894069104641v1, err: describe collection 431074894069104641 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431074894069104641"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_12_431028197852971009v0] [error="describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_12_431028197852971009v0] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_12_431028197852971009v0, err: describe collection 431028197852971009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028197852971009"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_1_431076965393629185v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_1_431076965393629185v1, err: describe collection 431076965393629185 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431076965393629185"]
[2022/02/09 17:50:15.562 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_39_431042895297118209v1] [error="describe collection 431042895297118209 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431042895297118209"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_39_431042895297118209v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_39_431042895297118209v1, err: describe collection 431042895297118209 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431042895297118209"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_32_431029110114091009v0] [error="describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"]
[2022/02/09 17:50:15.562 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_32_431029110114091009v0] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_32_431029110114091009v0, err: describe collection 431029110114091009 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029110114091009"]
[2022/02/09 17:50:15.563 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431029027971268609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029027971268609"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_31_431029027971268609v1] [error="describe collection 431029027971268609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029027971268609"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_31_431029027971268609v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_31_431029027971268609v1, err: describe collection 431029027971268609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029027971268609"]
[2022/02/09 17:50:15.563 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431028181783543809 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028181783543809"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_10_431028181783543809v0] [error="describe collection 431028181783543809 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028181783543809"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_10_431028181783543809v0] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_10_431028181783543809v0, err: describe collection 431028181783543809 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431028181783543809"]
[2022/02/09 17:50:15.563 +00:00] [ERROR] [segment_replica.go:660] ["Grpc error"] [error="describe collection 431029268029636609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029268029636609"] [stack="github.com/milvus-io/milvus/internal/datanode.(SegmentReplica).getCollectionSchema\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:660\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).initPKBloomFilter\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:417\ngithub.com/milvus-io/milvus/internal/datanode.(SegmentReplica).addFlushedSegment\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/segment_replica.go:398\ngithub.com/milvus-io/milvus/internal/datanode.(dataSyncService).initNodes\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:191\ngithub.com/milvus-io/milvus/internal/datanode.newDataSyncService\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_sync_service.go:88\ngithub.com/milvus-io/milvus/internal/datanode.(flowgraphManager).addAndStart\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/flow_graph_manager.go:57\ngithub.com/milvus-io/milvus/internal/datanode.(DataNode).handlePutEvent\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/data_node.go:374\ngithub.com/milvus-io/milvus/internal/datanode.(*channelEventManager).Run.func1\n\t/go/src/github.com/milvus-io/milvus/internal/datanode/event_manager.go:77"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [flow_graph_manager.go:59] ["new data sync service fail"] [vChannelName=by-dev-rootcoord-dml_35_431029268029636609v1] [error="describe collection 431029268029636609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029268029636609"]
[2022/02/09 17:50:15.563 +00:00] [WARN] [event_manager.go:58] ["handle put event fail, starting retry"] [vChanName=by-dev-rootcoord-dml_35_431029268029636609v1] ["retry interval"=20s] [error="fail to add and start flowgraph for vChanName: by-dev-rootcoord-dml_35_431029268029636609v1, err: describe collection 431029268029636609 from rootcoord wrong: DescribeCollection failed: can't find collection id : 431029268029636609"] `
From the logs seems that while you redeploy some of the states are not cleaned. please check the following: if your etcd is cleaned: by default is etcd.rootPath minio is cleaned properly: by default the bucket name is bucketName
I use docker-compose which is the default, and there is no customization and no error log for etcd and minio. So I don't know why I get those errors on the milvus-standalone container. It doesn't work even I remove all of the docker volumes and containers. I can use the milvus from scratch. What's your suggestion for a clean installation, is there something I missed?
etcd log:
{"level":"info","ts":"2022-02-10T04:29:07.218Z","caller":"mvcc/kvstore_compaction.go:57","msg":"finished scheduled compaction","compact-revision":576252,"took":"1.196835ms"} {"level":"info","ts":"2022-02-10T04:34:07.216Z","caller":"v3compactor/revision.go:86","msg":"starting auto revision compaction","revision":576552,"revision-compaction-retention":1000} {"level":"info","ts":"2022-02-10T04:34:07.218Z","caller":"v3compactor/revision.go:94","msg":"completed auto revision compaction","revision":576552,"revision-compaction-retention":1000,"took":"1.429718ms"} {"level":"info","ts":"2022-02-10T04:34:07.218Z","caller":"mvcc/index.go:214","msg":"compact tree index","revision":576552} {"level":"info","ts":"2022-02-10T04:34:07.220Z","caller":"mvcc/kvstore_compaction.go:57","msg":"finished scheduled compaction","compact-revision":576552,"took":"1.183081ms"}
minio log:
You are running an older version of MinIO released 1 year ago
Update: Run mc admin update
Endpoint: http://172.10.0.3:9000 http://127.0.0.1:9000
Browser Access:
http://172.10.0.3:9000 http://127.0.0.1:9000
Object API (Amazon S3 compatible):
Go: https://docs.min.io/docs/golang-client-quickstart-guide
Java: https://docs.min.io/docs/java-client-quickstart-guide
Python: https://docs.min.io/docs/python-client-quickstart-guide
JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
.NET: https://docs.min.io/docs/dotnet-client-quickstart-guide
Detected default credentials 'minioadmin:minioadmin', please change the credentials immediately using 'MINIO_ACCESS_KEY' and 'MINIO_SECRET_KEY'
Errors are no more. Removed all volumes after stopping Milvus. Now it has no issue for 200.000 ids. Thanks.
sudo rm -rf volumes
@ufuk23 is it okay to close this issue now? /assign @ufuk23
fixed
Is there an existing issue for this?
Environment
Current Behavior
When I search using expression like that "pk in [1, 2, 3, 4, 5, .............]", assume that 300.000 ids passed, I get the RPC error below.
RPC error: _execute_search_requests{'RPC start': '2022-02-09 13:05:19.965058', 'RPC error': '2022-02-09 13:05:22.613132'} RPC error: search{'RPC start': '2022-02-09 13:05:19.942964', 'RPC error': '2022-02-09 13:05:22.613132'} Traceback (most recent call last): File "C:\Users\Ufuk\PycharmProjects\featureExtraction\milvus2_search_test.py", line 96, in
result = art.search(vec, "embeddings", search_params, limit=10, expr=express, output_fields=["artifact_type"])
File "C:\Users\Ufuk\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\orm\collection.py", line 697, in search
res = conn.search(self._name, data, anns_field, param, limit, expr,
File "C:\Users\Ufuk\PycharmProjects\featureExtraction\venv\lib\site-packages\pymilvus\decorators.py", line 47, in handler
raise e
Expected Behavior
No error even if million ids passed in expression
Steps To Reproduce
Anything else?
No response