milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
27.28k stars 2.63k forks source link

[Bug]: After enabling dynamic schema, if there is no $meta field, the import of Parquet file format will fail. #32729

Closed zhuwenxing closed 1 week ago

zhuwenxing commented 3 weeks ago

Is there an existing issue for this?

Environment

- Milvus version:master
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

Even if the dynamic schema is enabled, users may not have dynamic columns in their data, so $meta can be optional.

Expected Behavior

failed_reason': "schema not equal, err=field '$meta' not in arrow schema: importing data failed: importing data failed", 'progress_percent': '0'}

Steps To Reproduce

No response

Milvus Log

failed ci job: https://jenkins.milvus.io:18080/blue/organizations/jenkins/Milvus%20HA%20CI/detail/PR-32706/3/pipeline/

Anything else?

No response

zhuwenxing commented 3 weeks ago

/assign @bigsheeper

yanliang567 commented 3 weeks ago

/unassign

bigsheeper commented 2 weeks ago

/assign @zhuwenxing /unassign

please help to verify