[Bug]: Use the Go client to delete the data. After a while, use attu's query interface to query the data. Only part of it was deleted.

dushulin commented 6 months ago

Is there an existing issue for this?

[X] I have searched the existing issues

Environment

- Milvus version:2.3.2
- Deployment mode(standalone or cluster):cluster
- MQ type(rocksmq, pulsar or kafka):    pulsar 
- SDK version(e.g. pymilvus v2.0.0rc2):go sdk v2.3.1
- OS(Ubuntu or CentOS): CentOS
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

some fileds： go code（delete func） after run this code, no a error info After a while, use attu's query interface to query the data. Only part of it was deleted.

Expected Behavior

All entities have been deleted and cannot be queried

Steps To Reproduce

No response

Milvus Log

No response

Anything else?

No response

dushulin commented 6 months ago

When the total amount of data to be deleted is more than 40, this problem will not occur. When the total amount of data is more than 200, this problem will occur.

yanliang567 commented 6 months ago

/assign @ThreadDao please help to reporduce the issue

/unassign

dushulin commented 6 months ago

信件已收到【自动回复】

SimFG commented 6 months ago

you can try to set the consistency_level param to Strong in the delete method

dushulin commented 6 months ago

you can try to set the consistency_level param to Strong in the delete method

how to call delete method by this param? I did not find api doc

dushulin commented 6 months ago

I saw a time traval value mentioned in your document. How long will it take before it will be retrieved by default?

SimFG commented 6 months ago

@dushulin maybe the go sdk have not give the param, you can set the collection consistency_level to strong, and try to delete the data.

dushulin commented 6 months ago

@dushulin maybe the go sdk have not give the param, you can set the collection consistency_level to strong, and try to delete the data. We need complex expr, so we can only set bounded，can we set consistency_level in query or search function? If not, can you team add this in go sdk?

SimFG commented 6 months ago

@congqixia please give this issue a look

SimFG commented 6 months ago

@dushulin you can set the consistency_level in search and query request https://github.com/milvus-io/milvus-sdk-go/blob/e697167f1828049c4c6237f0e3646b77b725ebdf/client/options.go#L174-L179

dushulin commented 6 months ago

@dushulin you can set the consistency_level in search and query request https://github.com/milvus-io/milvus-sdk-go/blob/e697167f1828049c4c6237f0e3646b77b725ebdf/client/options.go#L174-L179

Are there fields for curl?

SimFG commented 6 months ago

here is the client api

func (c *GrpcClient) Search(ctx context.Context, collName string, partitions []string,
    expr string, outputFields []string, vectors []entity.Vector, vectorField string, metricType entity.MetricType, topK int, sp entity.SearchParam, opts ...SearchQueryOptionFunc) ([]SearchResult, error) {

there is a SearchQueryOptionFunc param, it can be use the WithSearchQueryConsistencyLevel method to get this param

// WithSearchQueryConsistencyLevel specifies consistency level
func WithSearchQueryConsistencyLevel(cl entity.ConsistencyLevel) SearchQueryOptionFunc {
    return func(option *SearchQueryOption) {
        option.ConsistencyLevel = cl
    }
}

dushulin commented 6 months ago

here is the client api

func (c *GrpcClient) Search(ctx context.Context, collName string, partitions []string,
    expr string, outputFields []string, vectors []entity.Vector, vectorField string, metricType entity.MetricType, topK int, sp entity.SearchParam, opts ...SearchQueryOptionFunc) ([]SearchResult, error) {

there is a SearchQueryOptionFunc param, it can be use the WithSearchQueryConsistencyLevel method to get this param

// WithSearchQueryConsistencyLevel specifies consistency level
func WithSearchQueryConsistencyLevel(cl entity.ConsistencyLevel) SearchQueryOptionFunc {
  return func(option *SearchQueryOption) {
      option.ConsistencyLevel = cl
  }
}

I mean http api param name. ConsistencyLevel=Bounded? can you give me some example? use curl command

SimFG commented 6 months ago

@dushulin you can try to use the attu to delete the data. And the http api currently does not support this parameter, because it's a new parameter.

congqixia commented 5 months ago

sorry for the late reply. GoSDK use strong only for delete by expr.

phlamenco commented 5 months ago

sorry for the late reply. GoSDK use strong only for delete by expr.

current GoSDK delete function does not has a ConsistencyLevel parameter. Do you mean the delete function use strong ConsistencyLevel by default?

stale[bot] commented 4 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

dushulin commented 4 months ago

信件已收到【自动回复】

stale[bot] commented 3 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

dushulin commented 3 months ago

信件已收到【自动回复】

stale[bot] commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

dushulin commented 2 months ago

信件已收到【自动回复】

stale[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Reopen the issue with /reopen.

dushulin commented 1 month ago

信件已收到【自动回复】

milvus-io / milvus