scylladb / scylla-cluster-tests

Tests for Scylla Clusters
GNU Affero General Public License v3.0
56 stars 94 forks source link

run_stress_thread: self._stats['test_details'][section].append(cmd_params) AttributeError: 'dict' object has no attribute 'append' #9048

Open asias opened 9 hours ago

asias commented 9 hours ago

I used the following code to run c-s in my sct test.

    def _prepare_test_table2(self, ks='keyspace1', table=None):
        stress_cmd = "cassandra-stress write n=1000 cl=QUORUM -mode native cql3 " \
                     f"-schema 'replication(strategy=NetworkTopologyStrategy," \
                     f"replication_factor={self.tester.reliable_replication_factor})' -log interval=5"
        cs_thread = self.tester.run_stress_thread(
            stress_cmd=stress_cmd, keyspace_name=ks, stop_test_on_failure=False, round_robin=True)
        cs_thread.verify_results()

I got:

2024-10-24 07:41:01.802: (DisruptionEvent Severity.ERROR) period_type=end event_id=28e13ee4-7f6f-4536-95f9-54e9b31e00d4 duration=2s: nemesis_name=NoCorruptRepairAllNodesInParallel target_node=Node perf-latency-nemesis-ubuntu-db-node-61c231b2-1 [63.32.91.73 | 10.4.2.192] errors=ks10:
Traceback (most recent call last):
File "/home/ubuntu/scylla-cluster-tests/sdcm/utils/common.py", line 486, in run
result = future.result(time_out)
File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 458, in result
return self.__get_result()
File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/ubuntu/scylla-cluster-tests/sdcm/utils/common.py", line 462, in inner
return_val = fun(*args, **kwargs)
File "/home/ubuntu/scylla-cluster-tests/sdcm/nemesis.py", line 1721, in create_ks
self._prepare_test_table2(ks=ks, table='standard1')
File "/home/ubuntu/scylla-cluster-tests/sdcm/nemesis.py", line 2131, in _prepare_test_table2
cs_thread = self.tester.run_stress_thread(
File "/home/ubuntu/scylla-cluster-tests/sdcm/tester.py", line 1993, in run_stress_thread
return self.run_stress_cassandra_thread(**params)
File "/home/ubuntu/scylla-cluster-tests/sdcm/tester.py", line 2036, in run_stress_cassandra_thread
self.update_stress_cmd_details(stress_cmd, prefix, stresser="cassandra-stress",
File "/home/ubuntu/scylla-cluster-tests/sdcm/db_stats.py", line 710, in update_stress_cmd_details
self._stats['test_details'][section].append(cmd_params)
AttributeError: 'dict' object has no attribute 'append'

config:

n_monitor_nodes: 1
use_mgmt: false
store_perf_results: true
email_recipients: ["asias@scylladb.com"]
use_prepared_loaders: true
use_hdr_cs_histogram: true
use_placement_group: true
use_capacity_reservation: true
email_subject_postfix: 'latency during operations'

What is this? How can I avoid it.

Should I uses stats_aggregate_cmds=False?

        cs_thread = self.tester.run_stress_thread(
            stress_cmd=stress_cmd, keyspace_name=ks, stop_test_on_failure=False, round_robin=True, stats_aggregate_cmds=False)
soyacz commented 9 hours ago

@asias try using stats_aggregate_cmds=False - this should help

asias commented 8 hours ago

@asias try using stats_aggregate_cmds=False - this should help

Yes. I did that and it indeed helped.