milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
30.32k stars 2.91k forks source link

[Bug]: [partial load] fail to partial load 2 partitions with the same load_fields list #36037

Closed yanliang567 closed 4 weeks ago

yanliang567 commented 2 months ago

Is there an existing issue for this?

Environment

- Milvus version: master-20240903-6130a854
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):pymilvus-2.5.0rc72

Current Behavior

sometimes if the load_fields list is in different order, Milvus fail to partial load message=can't change the load field list for loaded collection: invalid parameter[expected=[100 105 103 101]][actual=[105 103 101 100]])>

Expected Behavior

can partial load successfully for 2 partitions if they have the same load_fields

Steps To Reproduce

1. create a collection with a few fields
2. create 2 partitions: p1 and p2
3. p1.load(load_fields=[pk_field.name, vector_field.name, load_string_field.name, load_int64_field.name])
4. p2.load(load_fields=[pk_field.name, vector_field.name, load_string_field.name, load_int64_field.name])

Milvus Log

No response

Anything else?

No response

yanliang567 commented 2 months ago

meanwhile, we need a way to list the load_fields to help users knows which fields is loaded at this moment. /assign @congqixia /unassign

congqixia commented 1 month ago

/assign @yanliang567 patch merged, could you please verify?

yanliang567 commented 1 month ago

will do

yanliang567 commented 4 weeks ago

verified and added tests in e2e