milvus-io / milvus

A cloud-native vector database, storage for next generation AI applications
https://milvus.io
Apache License 2.0
30.99k stars 2.95k forks source link

[Bug]: fail to parse the expression with templates if there are computings in the list #37451

Open yanliang567 opened 3 weeks ago

yanliang567 commented 3 weeks ago

Is there an existing issue for this?

Environment

- Milvus version: master-20241104-1b98bb42
- Deployment mode(standalone or cluster):
- MQ type(rocksmq, pulsar or kafka):    
- SDK version(e.g. pymilvus v2.0.0rc2): pymilvus-2.5.0rc106

Current Behavior

search fails with expression template: {"expr": f"{field} in {{value_0}}", "expr_params": {"value_0": [300/2, 900%40, -1030+800, 2048/2%200, (100+200)2]}} search_res = Error(code=65535, message=fail to search on QueryNode 1: worker(1) query failed: Operator::GetOutput failed for [Opera...== milvus::proto::plan::GenericValue::kFloatVal)" at /workspace/source/internal/core/src/exec/expression/Utils.h:130

while the same expression template works successfully for query()

Expected Behavior

works successfully

Steps To Reproduce

1. create a collection, insert some data with a float or int field
2. build index and load
3. search with the expression template

Milvus Log

No response

Anything else?

No response

yanliang567 commented 3 weeks ago

/assign @xiaocai2333 /unassign

xiaofan-luan commented 3 weeks ago

I don't think this is something we need to support. simply throw a reasonable error message is fine

xiaocai2333 commented 3 weeks ago

For Python, calculations within a list are automatically processed, so the expression you passed is effectively just {'expr': 'float in {value_0}', 'expr_params': {'value_0': [150.0, 20, 500, 24.0, 600]}}. The error occurs because the float field is not a display column and is considered a dynamic field (JSON column). Currently, JSON columns do not support combinations of different element types. For visible float columns, it will attempt to convert all elements to Double type, so it works. But for JSON columns, we retain the original types, so the elements here end up with different types (Int64 and Double).

And for JSON columns, we should support expressions like json['A'] in [1, 2.2, 'abc']. /assign @zhagnlu