Closed nairan-deshaw closed 6 days ago
@nairan-deshaw I think this was fixed by https://github.com/milvus-io/milvus/pull/34026, could you please upgrade to milvus 2.3.18 or 2.4.5 and retry? /assign @nairan-deshaw /unassign
Hi @yanliang567, we upgraded the cluster to 2.4.5 but this issue still persists. Partitions with large number of entries (over 1k) show this behaviour.
/assign @congqixia please help to take a look
I think i know the season: it is that Milvus only has 2 statuses: loaded and not_loaded. when the first partition is loaded, milvus will return immediately if calling to load the next partition. that's why this issue does not reporduce to the first partition, but the next ones. The same reason why it not reproduce if you call release partition in every loop. I think this is kind of by design @nairan-deshaw
@nairan-deshaw thanks for the feedback. There was a bug that newly loaded partition, of which collection or some other partitions is loaded before, is not guaranteed to be seen after being loaded. Patch is merged into 2.3&2.4 branch, you could wait for latest release. FYI.
@congqixia I do see that the bug is addressed in 2.3.19
release. Is it also addressed in the 2.4.6
release?
@congqixia I do see that the bug is addressed in
2.3.19
release. Is it also addressed in the2.4.6
release?
@nairan-deshaw just checked the commits in v2.4.6, this issue is not addressed. Since v2.4.6 is a hotfix for some other crucial bugs. It shall be released with v2.4.7. FYI.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Rotten issues close after 30d of inactivity. Reopen the issue with /reopen
.
This has been fixed from 2.4.7 onwards. Thanks for the help!
Closing
Is there an existing issue for this?
Environment
Current Behavior
When a partition is loaded the query to the partition takes a few seconds before results are returned. This happens when some partitions are already loaded into memory. The delay between loading finishing and availability of results is random. Adding a reproducer code:
If the partition is released after the
get
method is completed, the results are available immediately.Expected Behavior
The vectors should become available immediately after the partition load is completed.
Steps To Reproduce
Milvus Log
No response
Anything else?
We checked logs on the K8s ingress layer as well as the pod logs and both of seem to be normal. If there are any particular logs that need to be checked, please let us know.