wangting0128
commented
6 months ago flush timeout 180s
argo task：inverted-corn-1708358400 test case name：test_inverted_locust_partition_key_dml_standalone milvus image: master-20240219-43e8cd53-amd64
server：
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
inverted-corn-158400-2-82-4682-etcd-0 1/1 Running 0 3h17m 10.104.16.107 4am-node21 <none> <none>
inverted-corn-158400-2-82-4682-milvus-standalone-65b65977dnhdxg 1/1 Running 0 3h17m 10.104.27.229 4am-node31 <none> <none>
inverted-corn-158400-2-82-4682-minio-5bd6797c67-cdv4d 1/1 Running 0 3h17m 10.104.23.223 4am-node27 <none> <none>client pod name: inverted-corn-1708358400-3041780947 client logs:
test steps:
concurrent test and calculation of RT and QPS
:purpose: `partition_key: scalar enable partition_key(num_partitions=128)`
verify concurrent DML scenario which
scalar `id`(pk) & `int64_1` created INVERTED index and enable partition_key on `int64_1` field
:test steps:
1. create collection with fields:
'float_vector': 128dim,
'int64_1': is_partition_key
2. build indexes:
IVF_FLAT: 'float_vector'
INVERTED: 'id', 'int64_1'
3. insert 5 million data
4. flush collection
5. build indexes again using the same params
6. load collection
7. concurrent request:
- insert
- delete
- flush
- releasetest result:
{'server': {'deploy_tool': 'helm',
'deploy_mode': 'standalone',
'config_name': 'standalone_8c16m',
'config': {'standalone': {'resources': {'limits': {'cpu': '8.0',
'memory': '16Gi'},
'requests': {'cpu': '5.0',
'memory': '9Gi'}}},
'cluster': {'enabled': False},
'etcd': {'replicaCount': 1,
'metrics': {'enabled': True,
'podMonitor': {'enabled': True}}},
'minio': {'mode': 'standalone',
'metrics': {'podMonitor': {'enabled': True}}},
'pulsar': {'enabled': False},
'metrics': {'serviceMonitor': {'enabled': True}},
'log': {'level': 'debug'},
'image': {'all': {'repository': 'harbor.milvus.io/milvus/milvus',
'tag': 'master-20240219-43e8cd53-amd64'}}},
'host': 'inverted-corn-158400-2-82-4682-milvus.qa-milvus.svc.cluster.local',
'port': '19530',
'uri': ''},
'client': {'test_case_type': 'ConcurrentClientBase',
'test_case_name': 'test_inverted_locust_partition_key_dml_standalone',
'test_case_params': {'dataset_params': {'metric_type': 'L2',
'dim': 128,
'scalars_index': {'id': {'index_type': 'INVERTED'},
'int64_1': {'index_type': 'INVERTED'}},
'scalars_params': {'int64_1': {'params': {'is_partition_key': True}}},
'dataset_name': 'sift',
'dataset_size': 5000000,
'ni_per': 50000},
'collection_params': {'other_fields': ['int64_1'],
'shards_num': 2,
'num_partitions': 128},
'resource_groups_params': {'reset': False},
'database_user_params': {'reset_rbac': False,
'reset_db': False},
'index_params': {'index_type': 'IVF_FLAT',
'index_param': {'nlist': 1024}},
'concurrent_params': {'concurrent_number': 20,
'during_time': '3h',
'interval': 20,
'spawn_rate': None},
'concurrent_tasks': [{'type': 'insert',
'weight': 1,
'params': {'nb': 10,
'timeout': 180,
'random_id': True,
'random_vector': True,
'varchar_filled': False,
'start_id': 0}},
{'type': 'delete',
'weight': 1,
'params': {'expr': '',
'delete_length': 9,
'timeout': 30}},
{'type': 'flush',
'weight': 1,
'params': {'timeout': 180}},
{'type': 'release',
'weight': 1,
'params': {'timeout': 30}}]},
'run_id': 2024021985241562,
'datetime': '2024-02-19 16:02:04.509472',
'client_version': '2.4.0'},
'result': {'test_result': {'index': {'RT': 0.5101,
'id': {'RT': 0.5091},
'int64_1': {'RT': 0.5091}},
'insert': {'total_time': 145.5963,
'VPS': 34341.532,
'batch_time': 1.456,
'batch': 50000},
'flush': {'RT': 559.5905},
'load': {'RT': 2.5874},
'Locust': {'Aggregated': {'Requests': 4821,
'Fails': 1180,
'RPS': 0.45,
'fail_s': 0.24,
'RT_max': 182973.29,
'RT_avg': 44150.94,
'TP50': 27,
'TP99': 181000.0},
'delete': {'Requests': 1245,
'Fails': 0,
'RPS': 0.12,
'fail_s': 0.0,
'RT_max': 90.65,
'RT_avg': 6.5,
'TP50': 3,
'TP99': 56},
'flush': {'Requests': 1180,
'Fails': 1180,
'RPS': 0.11,
'fail_s': 1.0,
'RT_max': 182973.29,
'RT_avg': 180318.67,
'TP50': 180000.0,
'TP99': 181000.0},
'insert': {'Requests': 1202,
'Fails': 0,
'RPS': 0.11,
'fail_s': 0.0,
'RT_max': 280.76,
'RT_avg': 46.83,
'TP50': 41,
'TP99': 140.0},
'release': {'Requests': 1194,
'Fails': 0,
'RPS': 0.11,
'fail_s': 0.0,
'RT_max': 2203.0,
'RT_avg': 9.41,
'TP50': 2,
'TP99': 59}}}}}
yanliang567
longjiquan
XuanYang-cn
Is there an existing issue for this?
Environment
Current Behavior
argo task: multi-vector-scene-mix-84bqz test case name: test_hybrid_search_locust_dql_max_reqs_cluster
server:
client pod name: multi-vector-scene-mix-84bqz-3046920368 client log:
client.log
Expected Behavior
No response
Steps To Reproduce
Milvus Log
No response
Anything else?
server config:
client config:
test result: