milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
29.96k stars 2.87k forks source link

[Bug]: Collection load error #36350

Open xiaobai6480 opened 4 weeks ago

xiaobai6480 commented 4 weeks ago

Is there an existing issue for this?

Environment

- Milvus version:2.3.2
- Deployment mode(standalone or cluster):standalone 
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2): pymilvus 
- OS(Ubuntu or CentOS): CentOS 7.9
- CPU/Memory: 128G
- GPU: 
- Others:

Current Behavior

My milvus DB has Entites: 9,559,320, when I load all entries, it has an error

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

error.log

Anything else?

No response

yanliang567 commented 4 weeks ago

/assign @congqixia is it a known issue in 2.3.2? [2024/09/18 18:04:30.909 +00:00] [WARN] [segments/segment_loader.go:249] ["load segment failed when load data into memory"] [traceID=a6cfa43e943a28a856db5d67dcb7b5cf] [collectionID=445488778806039913] [segmentType=Sealed] [partitionID=445488778806039914] [segmentID=445488778806250241] [error="UnknownError: Error in GetObjectBuffer[errcode:-1, exception:, errmessage:curlCode: 18, Transferred a partial file, params:params, bucket=a-bucket, object=files/index_files/445488778811289235/1/445488778806039914/445488778806250241/IVF_FLAT_0]"] [errorVerbose="UnknownError: Error in GetObjectBuffer[errcode:-1, exception:, errmessage:curlCode: 18, Transferred a partial file, params:params, bucket=a-bucket, object=files/index_files/445488778811289235/1/445488778806039914/445488778806250241/IVF_FLAT_0]\

/unassign

xiaofan-luan commented 3 weeks ago

Is there an existing issue for this?

  • [x] I have searched the existing issues

Environment

- Milvus version:2.3.2
- Deployment mode(standalone or cluster):standalone 
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2): pymilvus 
- OS(Ubuntu or CentOS): CentOS 7.9
- CPU/Memory: 128G
- GPU: 
- Others:

Current Behavior

My milvus DB has Entites: 9,559,320, when I load all entries, it has an error

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

error.log

Anything else?

No response

I think 2.3.2 is too old. could you upgrade to 2.3.21 and retry?