[Bug]: Query failed with error `worker(8) query failed: output fields for IVF_SQ8 index is not allowed`

[zhuwenxing]

Is there an existing issue for this?

• [X] I have searched the existing issues

Environment

- Milvus version:master-20230610-dacf8b30
- Deployment mode(standalone or cluster):cluster
- MQ type(rocksmq, pulsar or kafka):pulsar    
- SDK version(e.g. pymilvus v2.0.0rc2):
- OS(Ubuntu or CentOS): 
- CPU/Memory: 
- GPU: 
- Others:

Current Behavior

[2023-06-11T09:43:01.289Z] 2023-06-11 09:43:01.028 | INFO     | MainThread |utils:load_and_search:207 - load collection

[2023-06-11T09:43:05.443Z] 2023-06-11 09:43:04.557 | INFO     | MainThread |utils:load_and_search:211 - load time: 3.5288

[2023-06-11T09:43:05.443Z] 2023-06-11 09:43:04.577 | INFO     | MainThread |utils:load_and_search:225 - {'metric_type': 'L2', 'params': {'nprobe': 10}}

[2023-06-11T09:43:05.443Z] 2023-06-11 09:43:04.577 | INFO     | MainThread |utils:load_and_search:228 - 

[2023-06-11T09:43:05.443Z] Search...

[2023-06-11T09:43:05.443Z] 2023-06-11 09:43:04.585 | INFO     | MainThread |utils:load_and_search:239 - hit: id: 2156, distance: 32.400264739990234, entity: {'random_value': -12.0, 'count': 2156}

[2023-06-11T09:43:05.443Z] 2023-06-11 09:43:04.585 | INFO     | MainThread |utils:load_and_search:239 - hit: id: 602, distance: 32.57061767578125, entity: {'random_value': -11.0, 'count': 602}

[2023-06-11T09:43:05.443Z] 2023-06-11 09:43:04.585 | INFO     | MainThread |utils:load_and_search:239 - hit: id: 1960, distance: 32.72724151611328, entity: {'random_value': -13.0, 'count': 1960}

[2023-06-11T09:43:05.443Z] 2023-06-11 09:43:04.585 | INFO     | MainThread |utils:load_and_search:239 - hit: id: 2766, distance: 32.757911682128906, entity: {'random_value': -14.0, 'count': 2766}

[2023-06-11T09:43:05.443Z] 2023-06-11 09:43:04.585 | INFO     | MainThread |utils:load_and_search:239 - hit: id: 2698, distance: 33.18340301513672, entity: {'random_value': -17.0, 'count': 2698}

[2023-06-11T09:43:05.444Z] 2023-06-11 09:43:04.585 | INFO     | MainThread |utils:load_and_search:242 - [2156, 602, 1960, 2766, 2698]

[2023-06-11T09:43:05.444Z] 2023-06-11 09:43:04.585 | INFO     | MainThread |utils:load_and_search:244 - search latency: 0.0075s

[2023-06-11T09:43:05.444Z] RPC error: [query], <MilvusException: (code=1, message=fail to query on all shard leaders, err=fail to Query, QueryNode ID = 4, reason=failed to query channel, err=worker(8) query failed: output fields for IVF_SQ8 index is not allowed)>, <Time:{'RPC start': '2023-06-11 09:43:04.585957', 'RPC error': '2023-06-11 09:43:04.592104'}>

[2023-06-11T09:43:05.444Z] Traceback (most recent call last):

[2023-06-11T09:43:05.444Z]   File "scripts/action_before_reinstall.py", line 48, in <module>

[2023-06-11T09:43:05.444Z]     task_2(data_size, host)

[2023-06-11T09:43:05.444Z]   File "scripts/action_before_reinstall.py", line 34, in task_2

[2023-06-11T09:43:05.444Z]     load_and_search(prefix)

[2023-06-11T09:43:05.444Z]   File "/home/jenkins/agent/workspace/tests/python_client/deploy/scripts/utils.py", line 248, in load_and_search

[2023-06-11T09:43:05.444Z]     res = c.query(expr, output_fields, timeout=120)

[2023-06-11T09:43:05.444Z]   File "/usr/local/lib/python3.8/dist-packages/pymilvus/orm/collection.py", line 774, in query

[2023-06-11T09:43:05.444Z]     res = conn.query(self._name, expr, output_fields, partition_names,

[2023-06-11T09:43:05.444Z]   File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 109, in handler

[2023-06-11T09:43:05.444Z]     raise e

[2023-06-11T09:43:05.444Z]   File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 105, in handler

[2023-06-11T09:43:05.444Z]     return func(*args, **kwargs)

[2023-06-11T09:43:05.444Z]   File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 136, in handler

[2023-06-11T09:43:05.444Z]     ret = func(self, *args, **kwargs)

[2023-06-11T09:43:05.444Z]   File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 85, in handler

[2023-06-11T09:43:05.444Z]     raise e

[2023-06-11T09:43:05.444Z]   File "/usr/local/lib/python3.8/dist-packages/pymilvus/decorators.py", line 50, in handler

[2023-06-11T09:43:05.444Z]     return func(self, *args, **kwargs)

[2023-06-11T09:43:05.444Z]   File "/usr/local/lib/python3.8/dist-packages/pymilvus/client/grpc_handler.py", line 1045, in query

[2023-06-11T09:43:05.444Z]     raise MilvusException(response.status.error_code, response.status.reason)

[2023-06-11T09:43:05.444Z] pymilvus.exceptions.MilvusException: <MilvusException: (code=1, message=fail to query on all shard leaders, err=fail to Query, QueryNode ID = 4, reason=failed to query channel, err=worker(8) query failed: output fields for IVF_SQ8 index is not allowed)>

script returned exit code 1

Expected Behavior

No response

Steps To Reproduce

No response

Milvus Log

failed job: https://qa-jenkins.milvus.io/blue/organizations/jenkins/deploy_test_cron/detail/deploy_test_cron/900/pipeline log:

artifacts-pulsar-cluster-reinstall-900-server-logs.tar.gz

artifacts-pulsar-cluster-reinstall-900-pytest-logs.tar.gz

Anything else?

No response

[xiaofan-luan]

this is as expected, faiss don't support outfield for SQ and PQ

[zhuwenxing]

this is as expected, faiss don't support outfield for SQ and PQ

I think the error message could be better, like vector field in output_field for IVF_SQ8 index is not allowed

[bigsheeper]

/assign

[yanliang567]

/unassign

[zhuwenxing]

@bigsheeper

in the reinstallation test, the vector field can be output for IVF sq8 in the first deployment, but it raises error after reinstallation.

failed job: https://qa-jenkins.milvus.io/blue/organizations/jenkins/deploy_test_kafka_cron/detail/deploy_test_kafka_cron/897/pipeline log:

artifacts-kafka-standalone-reinstall-897-pytest-logs.tar.gz

artifacts-kafka-standalone-reinstall-897-server-first-deployment-logs.tar.gz

artifacts-kafka-standalone-reinstall-897-server-second-deployment-logs.tar.gz

[bigsheeper]

/assign @zhuwenxing This issue no longer reproduce. Its previous occurrence was most likely due to search on streaming segment, which inherently supports get raw data.

[bigsheeper]

/unassign

[caoping6]

How did it solve? I also encountered this problem

[yanliang567]

retrieving vector will be supported for all index types and metric types in Milvus v2.3.2, which would be released soon this week or next.