search

egeberkaygulcan / dstest

2 stars 1 forks source link

etcd example in configs return an empty schedule in output #30

Open idilkara opened 1 month ago

idilkara commented 1 month ago

Im trying to run the etcd example in the configs but for the iterations that finds a bug candidate, the schedule in the output file is empty. For instance, for the following input and output, the schedule.log files are empty.

Input: docker run --rm -v ./configs:/configs -v ./output:/root/dstest/output egeberkaygulcan/dstest run -c /configs/etcd.yml Output:

ℹ tracking data in /tmp/stats.3320328101.json config: /configs/etcd.yml Starting dstest Name: etcd-test [TestEngine] 2024/07/22 13:50:29 Starting experiment 1... [TestEngine] 2024/07/22 13:50:29 Starting iteration 1 [NetworkManager] 2024/07/22 13:50:29 Network manager initialized Faults: [NetworkManager] 2024/07/22 13:50:29 Network manager running [Worker 2] 2024/07/22 13:50:29 Running worker with: /root/dstest/scripts/etcd_server.sh 10000 10001 6002 n2 [Worker 0] 2024/07/22 13:50:29 Running worker with: /root/dstest/scripts/etcd_server.sh 6000 10001 10002 n0 [Worker 1] 2024/07/22 13:50:29 Running worker with: /root/dstest/scripts/etcd_server.sh 10000 6001 10002 n1 [Worker 2] 2024/07/22 13:50:39 Timeout, killing process. [Worker 2] 2024/07/22 13:50:39 Killed worker 2 [Worker 2] 2024/07/22 13:50:39 Calling the clean script. [Worker 0] 2024/07/22 13:50:39 Timeout, killing process. [Worker 1] 2024/07/22 13:50:39 Timeout, killing process. [Worker 1] 2024/07/22 13:50:39 Killed worker 1 [Worker 1] 2024/07/22 13:50:39 Calling the clean script. [Worker 0] 2024/07/22 13:50:39 Killed worker 0 [Worker 0] 2024/07/22 13:50:39 Calling the clean script. [ProcessManager]2024/07/22 13:50:39 Worker 0 status: Timeout [ProcessManager]2024/07/22 13:50:39 Worker 1 status: Timeout [ProcessManager]2024/07/22 13:50:39 Worker 2 status: Timeout [ProcessManager]2024/07/22 13:50:39 Found bug candidate at iteration 0 [TestEngine] 2024/07/22 13:50:39 Shutting down ProcessManager... [TestEngine] 2024/07/22 13:50:39 Shutting down NetworkManager... [TestEngine] 2024/07/22 13:50:39 Shutdown complete. [TestEngine] 2024/07/22 13:50:39 Checking for bugs... [TestEngine] 2024/07/22 13:50:39 Iteration complete. [TestEngine] 2024/07/22 13:50:39 Starting iteration 2 [NetworkManager] 2024/07/22 13:50:39 Network manager initialized Faults: [Worker 2] 2024/07/22 13:50:39 Running worker with: /root/dstest/scripts/etcd_server.sh 10000 10001 6002 n2 [Worker 1] 2024/07/22 13:50:39 Running worker with: /root/dstest/scripts/etcd_server.sh 10000 6001 10002 n1 [NetworkManager] 2024/07/22 13:50:39 Network manager running [Worker 0] 2024/07/22 13:50:39 Running worker with: /root/dstest/scripts/etcd_server.sh 6000 10001 10002 n0 [Worker 2] 2024/07/22 13:50:49 Timeout, killing process. [Worker 2] 2024/07/22 13:50:49 Killed worker 2 [Worker 2] 2024/07/22 13:50:49 Calling the clean script. [Worker 1] 2024/07/22 13:50:49 Timeout, killing process. [Worker 1] 2024/07/22 13:50:49 Killed worker 1 [Worker 1] 2024/07/22 13:50:49 Calling the clean script. [Worker 0] 2024/07/22 13:50:49 Timeout, killing process. [Worker 0] 2024/07/22 13:50:49 Killed worker 0 [Worker 0] 2024/07/22 13:50:49 Calling the clean script. [ProcessManager]2024/07/22 13:50:49 Worker 0 status: Timeout [ProcessManager]2024/07/22 13:50:49 Worker 1 status: Timeout [ProcessManager]2024/07/22 13:50:49 Worker 2 status: Timeout [ProcessManager]2024/07/22 13:50:49 Found bug candidate at iteration 1 [TestEngine] 2024/07/22 13:50:49 Shutting down ProcessManager... [TestEngine] 2024/07/22 13:50:49 Shutting down NetworkManager... [TestEngine] 2024/07/22 13:50:49 Shutdown complete. [TestEngine] 2024/07/22 13:50:49 Checking for bugs... [TestEngine] 2024/07/22 13:50:49 Iteration complete. [TestEngine] 2024/07/22 13:50:49 Starting iteration 3 [NetworkManager] 2024/07/22 13:50:49 Network manager initialized Faults: [Worker 2] 2024/07/22 13:50:49 Running worker with: /root/dstest/scripts/etcd_server.sh 10000 10001 6002 n2 [Worker 0] 2024/07/22 13:50:49 Running worker with: /root/dstest/scripts/etcd_server.sh 6000 10001 10002 n0 [NetworkManager] 2024/07/22 13:50:49 Network manager running [Worker 1] 2024/07/22 13:50:49 Running worker with: /root/dstest/scripts/etcd_server.sh 10000 6001 10002 n1 [Worker 2] 2024/07/22 13:50:59 Timeout, killing process. [Worker 1] 2024/07/22 13:50:59 Timeout, killing process. [Worker 1] 2024/07/22 13:50:59 Killed worker 1 [Worker 0] 2024/07/22 13:50:59 Timeout, killing process. [Worker 1] 2024/07/22 13:50:59 Calling the clean script. [Worker 0] 2024/07/22 13:50:59 Killed worker 0 [Worker 0] 2024/07/22 13:50:59 Calling the clean script. [Worker 2] 2024/07/22 13:50:59 Killed worker 2 [Worker 2] 2024/07/22 13:50:59 Calling the clean script. [ProcessManager]2024/07/22 13:50:59 Worker 0 status: Timeout [ProcessManager]2024/07/22 13:50:59 Worker 1 status: Timeout [ProcessManager]2024/07/22 13:50:59 Worker 2 status: Timeout [ProcessManager]2024/07/22 13:50:59 Found bug candidate at iteration 2 [TestEngine] 2024/07/22 13:50:59 Shutting down ProcessManager... [TestEngine] 2024/07/22 13:50:59 Shutting down NetworkManager... [TestEngine] 2024/07/22 13:50:59 Shutdown complete. [TestEngine] 2024/07/22 13:50:59 Checking for bugs... [TestEngine] 2024/07/22 13:50:59 Iteration complete. [TestEngine] 2024/07/22 13:50:59 Starting iteration 4 [NetworkManager] 2024/07/22 13:50:59 Network manager initialized Faults: [NetworkManager] 2024/07/22 13:50:59 Network manager running [Worker 2] 2024/07/22 13:50:59 Running worker with: /root/dstest/scripts/etcd_server.sh 10000 10001 6002 n2 [Worker 1] 2024/07/22 13:50:59 Running worker with: /root/dstest/scripts/etcd_server.sh 10000 6001 10002 n1 [Worker 0] 2024/07/22 13:50:59 Running worker with: /root/dstest/scripts/etcd_server.sh 6000 10001 10002 n0 [Worker 2] 2024/07/22 13:51:09 Timeout, killing process. [Worker 1] 2024/07/22 13:51:09 Timeout, killing process. [Worker 1] 2024/07/22 13:51:09 Killed worker 1 [Worker 1] 2024/07/22 13:51:09 Calling the clean script. [Worker 2] 2024/07/22 13:51:09 Killed worker 2 [Worker 2] 2024/07/22 13:51:09 Calling the clean script. [Worker 0] 2024/07/22 13:51:09 Timeout, killing process. [Worker 0] 2024/07/22 13:51:09 Killed worker 0 [Worker 0] 2024/07/22 13:51:09 Calling the clean script. [ProcessManager]2024/07/22 13:51:09 Worker 0 status: Timeout [ProcessManager]2024/07/22 13:51:09 Worker 1 status: Timeout [ProcessManager]2024/07/22 13:51:09 Worker 2 status: Timeout [ProcessManager]2024/07/22 13:51:09 Found bug candidate at iteration 3 [TestEngine] 2024/07/22 13:51:09 Shutting down ProcessManager... [TestEngine] 2024/07/22 13:51:09 Shutting down NetworkManager... [TestEngine] 2024/07/22 13:51:09 Shutdown complete. [TestEngine] 2024/07/22 13:51:09 Checking for bugs... [TestEngine] 2024/07/22 13:51:09 Iteration complete. [TestEngine] 2024/07/22 13:51:09 Starting iteration 5 [NetworkManager] 2024/07/22 13:51:09 Network manager initialized Faults: [Worker 2] 2024/07/22 13:51:09 Running worker with: /root/dstest/scripts/etcd_server.sh 10000 10001 6002 n2 [NetworkManager] 2024/07/22 13:51:09 Network manager running [Worker 0] 2024/07/22 13:51:09 Running worker with: /root/dstest/scripts/etcd_server.sh 6000 10001 10002 n0 [Worker 1] 2024/07/22 13:51:09 Running worker with: /root/dstest/scripts/etcd_server.sh 10000 6001 10002 n1 [Worker 2] 2024/07/22 13:51:19 Timeout, killing process. [Worker 0] 2024/07/22 13:51:19 Timeout, killing process. [Worker 0] 2024/07/22 13:51:19 Killed worker 0 [Worker 1] 2024/07/22 13:51:19 Timeout, killing process. [Worker 0] 2024/07/22 13:51:19 Calling the clean script. [Worker 1] 2024/07/22 13:51:19 Killed worker 1 [Worker 1] 2024/07/22 13:51:19 Calling the clean script. [Worker 2] 2024/07/22 13:51:19 Killed worker 2 [Worker 2] 2024/07/22 13:51:19 Calling the clean script. [ProcessManager]2024/07/22 13:51:19 Worker 0 status: Timeout [ProcessManager]2024/07/22 13:51:19 Worker 1 status: Timeout [ProcessManager]2024/07/22 13:51:19 Worker 2 status: Timeout [ProcessManager]2024/07/22 13:51:19 Found bug candidate at iteration 4 [TestEngine] 2024/07/22 13:51:19 Shutting down ProcessManager... [TestEngine] 2024/07/22 13:51:19 Shutting down NetworkManager... [TestEngine] 2024/07/22 13:51:19 Shutdown complete. [TestEngine] 2024/07/22 13:51:19 Checking for bugs... [TestEngine] 2024/07/22 13:51:19 Iteration complete.

idilkara commented 1 month ago

Replica params used in the etcd.yml are as:

**ReplicaParams:

I thought it might have been a problem but I changed it to this,

ReplicaParams:

and tried again.

This time I get a similar output with the schedule.log files empty again.

Output:

ℹ tracking data in /tmp/stats.338720227.json config: /configs/etcd.yml Starting dstest Name: etcd-test [TestEngine] 2024/07/22 13:59:41 Starting experiment 1... [TestEngine] 2024/07/22 13:59:41 Starting iteration 1 [NetworkManager] 2024/07/22 13:59:41 Network manager initialized Faults: [NetworkManager] 2024/07/22 13:59:41 Network manager running [Worker 2] 2024/07/22 13:59:41 Running worker with: /root/dstest/scripts/etcd_server.sh 10006 10007 6002 n2 [Worker 1] 2024/07/22 13:59:41 Running worker with: /root/dstest/scripts/etcd_server.sh 10003 6001 10005 n1 [Worker 0] 2024/07/22 13:59:41 Running worker with: /root/dstest/scripts/etcd_server.sh 6000 10001 10002 n0 [Worker 1] 2024/07/22 13:59:51 Timeout, killing process. [Worker 1] 2024/07/22 13:59:51 Killed worker 1 [Worker 1] 2024/07/22 13:59:51 Calling the clean script. [Worker 2] 2024/07/22 13:59:51 Timeout, killing process. [Worker 0] 2024/07/22 13:59:51 Timeout, killing process. [Worker 0] 2024/07/22 13:59:51 Killed worker 0 [Worker 0] 2024/07/22 13:59:51 Calling the clean script. [Worker 2] 2024/07/22 13:59:51 Killed worker 2 [Worker 2] 2024/07/22 13:59:51 Calling the clean script. [ProcessManager]2024/07/22 13:59:51 Worker 0 status: Timeout [ProcessManager]2024/07/22 13:59:51 Worker 1 status: Timeout [ProcessManager]2024/07/22 13:59:51 Worker 2 status: Timeout [ProcessManager]2024/07/22 13:59:51 Found bug candidate at iteration 0 [TestEngine] 2024/07/22 13:59:51 Shutting down ProcessManager... [TestEngine] 2024/07/22 13:59:51 Shutting down NetworkManager... [TestEngine] 2024/07/22 13:59:51 Shutdown complete. [TestEngine] 2024/07/22 13:59:51 Checking for bugs... [TestEngine] 2024/07/22 13:59:51 Iteration complete. [TestEngine] 2024/07/22 13:59:51 Starting iteration 2 [NetworkManager] 2024/07/22 13:59:51 Network manager initialized Faults: [Worker 2] 2024/07/22 13:59:51 Running worker with: /root/dstest/scripts/etcd_server.sh 10006 10007 6002 n2 [Worker 0] 2024/07/22 13:59:51 Running worker with: /root/dstest/scripts/etcd_server.sh 6000 10001 10002 n0 [NetworkManager] 2024/07/22 13:59:51 Network manager running [Worker 1] 2024/07/22 13:59:51 Running worker with: /root/dstest/scripts/etcd_server.sh 10003 6001 10005 n1 [Worker 0] 2024/07/22 14:00:01 Timeout, killing process. [Worker 2] 2024/07/22 14:00:01 Timeout, killing process. [Worker 2] 2024/07/22 14:00:01 Killed worker 2 [Worker 2] 2024/07/22 14:00:01 Calling the clean script. [Worker 0] 2024/07/22 14:00:01 Killed worker 0 [Worker 0] 2024/07/22 14:00:01 Calling the clean script. [Worker 1] 2024/07/22 14:00:01 Timeout, killing process. [Worker 1] 2024/07/22 14:00:01 Killed worker 1 [Worker 1] 2024/07/22 14:00:01 Calling the clean script. [ProcessManager]2024/07/22 14:00:01 Worker 0 status: Timeout [ProcessManager]2024/07/22 14:00:01 Worker 1 status: Timeout [ProcessManager]2024/07/22 14:00:01 Worker 2 status: Timeout [ProcessManager]2024/07/22 14:00:01 Found bug candidate at iteration 1 [TestEngine] 2024/07/22 14:00:01 Shutting down ProcessManager... [TestEngine] 2024/07/22 14:00:01 Shutting down NetworkManager... [TestEngine] 2024/07/22 14:00:01 Shutdown complete. [TestEngine] 2024/07/22 14:00:01 Checking for bugs... [TestEngine] 2024/07/22 14:00:01 Iteration complete. [TestEngine] 2024/07/22 14:00:01 Starting iteration 3 Faults: [NetworkManager] 2024/07/22 14:00:01 Network manager initialized [NetworkManager] 2024/07/22 14:00:01 Network manager running [Worker 2] 2024/07/22 14:00:01 Running worker with: /root/dstest/scripts/etcd_server.sh 10006 10007 6002 n2 [Worker 0] 2024/07/22 14:00:01 Running worker with: /root/dstest/scripts/etcd_server.sh 6000 10001 10002 n0 [Worker 1] 2024/07/22 14:00:01 Running worker with: /root/dstest/scripts/etcd_server.sh 10003 6001 10005 n1 [Worker 1] 2024/07/22 14:00:11 Timeout, killing process. [Worker 2] 2024/07/22 14:00:11 Timeout, killing process. [Worker 1] 2024/07/22 14:00:11 Killed worker 1 [Worker 1] 2024/07/22 14:00:11 Calling the clean script. [Worker 0] 2024/07/22 14:00:11 Timeout, killing process. [Worker 2] 2024/07/22 14:00:11 Killed worker 2 [Worker 2] 2024/07/22 14:00:11 Calling the clean script. [Worker 0] 2024/07/22 14:00:11 Killed worker 0 [Worker 0] 2024/07/22 14:00:11 Calling the clean script. [ProcessManager]2024/07/22 14:00:11 Worker 0 status: Timeout [ProcessManager]2024/07/22 14:00:11 Worker 1 status: Timeout [ProcessManager]2024/07/22 14:00:11 Worker 2 status: Timeout [ProcessManager]2024/07/22 14:00:11 Found bug candidate at iteration 2 [TestEngine] 2024/07/22 14:00:11 Shutting down ProcessManager... [TestEngine] 2024/07/22 14:00:11 Shutting down NetworkManager... [TestEngine] 2024/07/22 14:00:11 Shutdown complete. [TestEngine] 2024/07/22 14:00:11 Checking for bugs... [TestEngine] 2024/07/22 14:00:11 Iteration complete. [TestEngine] 2024/07/22 14:00:11 Starting iteration 4 [NetworkManager] 2024/07/22 14:00:11 Network manager initialized Faults: [NetworkManager] 2024/07/22 14:00:11 Network manager running [Worker 2] 2024/07/22 14:00:11 Running worker with: /root/dstest/scripts/etcd_server.sh 10006 10007 6002 n2 [Worker 0] 2024/07/22 14:00:11 Running worker with: /root/dstest/scripts/etcd_server.sh 6000 10001 10002 n0 [Worker 1] 2024/07/22 14:00:11 Running worker with: /root/dstest/scripts/etcd_server.sh 10003 6001 10005 n1 [Worker 0] 2024/07/22 14:00:21 Timeout, killing process. [Worker 0] 2024/07/22 14:00:21 Killed worker 0 [Worker 0] 2024/07/22 14:00:21 Calling the clean script. [Worker 2] 2024/07/22 14:00:21 Timeout, killing process. [Worker 1] 2024/07/22 14:00:21 Timeout, killing process. [Worker 2] 2024/07/22 14:00:21 Killed worker 2 [Worker 1] 2024/07/22 14:00:21 Killed worker 1 [Worker 2] 2024/07/22 14:00:21 Calling the clean script. [Worker 1] 2024/07/22 14:00:21 Calling the clean script. [ProcessManager]2024/07/22 14:00:21 Worker 0 status: Timeout [ProcessManager]2024/07/22 14:00:21 Worker 1 status: Timeout [ProcessManager]2024/07/22 14:00:21 Worker 2 status: Timeout [ProcessManager]2024/07/22 14:00:21 Found bug candidate at iteration 3 [TestEngine] 2024/07/22 14:00:21 Shutting down ProcessManager... [TestEngine] 2024/07/22 14:00:21 Shutting down NetworkManager... [TestEngine] 2024/07/22 14:00:21 Shutdown complete. [TestEngine] 2024/07/22 14:00:21 Checking for bugs... [TestEngine] 2024/07/22 14:00:21 Iteration complete. [TestEngine] 2024/07/22 14:00:21 Starting iteration 5 [NetworkManager] 2024/07/22 14:00:21 Network manager initialized Faults: [Worker 2] 2024/07/22 14:00:21 Running worker with: /root/dstest/scripts/etcd_server.sh 10006 10007 6002 n2 [Worker 0] 2024/07/22 14:00:21 Running worker with: /root/dstest/scripts/etcd_server.sh 6000 10001 10002 n0 [NetworkManager] 2024/07/22 14:00:21 Network manager running [Worker 1] 2024/07/22 14:00:21 Running worker with: /root/dstest/scripts/etcd_server.sh 10003 6001 10005 n1 [Worker 0] 2024/07/22 14:00:31 Timeout, killing process. [Worker 1] 2024/07/22 14:00:31 Timeout, killing process. [Worker 0] 2024/07/22 14:00:31 Killed worker 0 [Worker 0] 2024/07/22 14:00:31 Calling the clean script. [Worker 2] 2024/07/22 14:00:31 Timeout, killing process. [Worker 1] 2024/07/22 14:00:31 Killed worker 1 [Worker 2] 2024/07/22 14:00:31 Killed worker 2 [Worker 2] 2024/07/22 14:00:31 Calling the clean script. [Worker 1] 2024/07/22 14:00:31 Calling the clean script. [ProcessManager]2024/07/22 14:00:31 Worker 0 status: Timeout [ProcessManager]2024/07/22 14:00:31 Worker 1 status: Timeout [ProcessManager]2024/07/22 14:00:31 Worker 2 status: Timeout [ProcessManager]2024/07/22 14:00:31 Found bug candidate at iteration 4 [TestEngine] 2024/07/22 14:00:31 Shutting down ProcessManager... [TestEngine] 2024/07/22 14:00:31 Shutting down NetworkManager... [TestEngine] 2024/07/22 14:00:31 Shutdown complete. [TestEngine] 2024/07/22 14:00:31 Checking for bugs... [TestEngine] 2024/07/22 14:00:31 Iteration complete.