Scylla nodes should be started sequentially (start node N + 1 only after N has finished bootstrapping). Currently, Terraform starts Scylla on all nodes concurrently (systemctl start). Similarly, nodes_start and restart_cluster starts all the nodes in parallel. Starting the nodes in parallel may lead to a data loss or subpar CDC performance, as I learned the hard way in this issue (see that comment for larger context).
Scylla nodes should be started sequentially (start node
N + 1only afterNhas finished bootstrapping). Currently, Terraform starts Scylla on all nodes concurrently (systemctl start). Similarly,nodes_startandrestart_clusterstarts all the nodes in parallel. Starting the nodes in parallel may lead to a data loss or subpar CDC performance, as I learned the hard way in this issue (see that comment for larger context).