Closed NicoYuan1986 closed 5 days ago
- Milvus version: master-20240910-60f4db2f - Deployment mode(standalone or cluster): standalone - MQ type(rocksmq, pulsar or kafka): rocksmq - SDK version(e.g. pymilvus v2.0.0rc2): - OS(Ubuntu or CentOS): - CPU/Memory: - GPU: - Others:
Milvus standalone will crash when open all mmap on vector index and field, scalar index and field.
failed cases: https://qa-jenkins.milvus.io/blue/organizations/jenkins/existing-milvus/detail/existing-milvus/4/pipeline/
milvus pod: (on devops chaos-testing)
kubectl get pods |grep func-mmap-standalone-vlzph func-mmap-standalone-vlzph-etcd-0 1/1 Running 0 4h16m func-mmap-standalone-vlzph-milvus-standalone-7d468974fd-jj82q 0/1 CrashLoopBackOff 45 (28s ago) 4h16m func-mmap-standalone-vlzph-minio-9c5d55498-w82q7 1/1 Running 0 4h16m
pass
No response
https://grafana-devops.zilliz.cc/explore?orgId=1&left=%7B%22datasource%22:%22Loki%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bpod%3D%5C%22func-mmap-standalone-vlzph-milvus-standalone-7d468974fd-jj82q%5C%22%7D%22%7D%5D,%22range%22:%7B%22from%22:%22now-3h%22,%22to%22:%22now%22%7D%7D
/assign @sunby /unassign
I retest twice. not reproduce.
milvus standalone did not crash. But it still exists load timeout error.
what is the progress for now?
not reproduced.
Is there an existing issue for this?
Environment
Current Behavior
Milvus standalone will crash when open all mmap on vector index and field, scalar index and field.
failed cases: https://qa-jenkins.milvus.io/blue/organizations/jenkins/existing-milvus/detail/existing-milvus/4/pipeline/
milvus pod: (on devops chaos-testing)
Expected Behavior
pass
Steps To Reproduce
No response
Milvus Log
https://grafana-devops.zilliz.cc/explore?orgId=1&left=%7B%22datasource%22:%22Loki%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22expr%22:%22%7Bpod%3D%5C%22func-mmap-standalone-vlzph-milvus-standalone-7d468974fd-jj82q%5C%22%7D%22%7D%5D,%22range%22:%7B%22from%22:%22now-3h%22,%22to%22:%22now%22%7D%7D
Anything else?
No response