openshift / svt

Apache License 2.0
124 stars 105 forks source link

Add pre and post tasks #752

Closed qiliRedHat closed 1 year ago

qiliRedHat commented 1 year ago
  1. Add pre_tasks and post_tasks to run before and after the looped tasks, only once, to do some preparation and cleanup work. https://issues.redhat.com/browse/OCPQE-14484
  2. Updated the example_reliability.yaml file to include the pre_tasks and post_tasks
  3. Merged the scale_up and scale_down funcs to a unified scale_deployment func, add a __check_replicas func to check replicas reach the expected number
  4. Replace 'persona' with 'user_name' to make it easier to understand and compatible to other customized user password files
  5. Updated the README.md to include the new changes and to a more organized order.
qiliRedHat commented 1 year ago

Test config

  groups:
    - name: admin
      user_name: kubeadmin
      loops: forever
      trigger: 600
      jitter: 60
      interval: 10
      tasks:
        - func check_operators
    - name: dev-app-prod
      user_name: testuser-
      user_start: 15
      user_end: 17
      loops: forever
      trigger: 10
      jitter: 0
      interval: 10
      pre_tasks:
          - func delete_all_projects # clear all projects
          - func new_project 2 # new 2 projects
          - func new_app 2 # new app in 2 namespaces
      tasks:
        - func load_app 2 10 # load apps in 2 namespaces with 10 clients for each
        - func scale_deployment 2 2 # scale app in 2 namespaces to 2 replicas
        - func scale_deployment 2 1 # scale app in 2 namespaces to 1 replicas
      post_tasks:
        - func delete_project 2

Pre tasks run only once for each user

2023-03-21 10:20:56,425 - INFO - '[User: testuser-15] [Task: func delete_all_projects]: finished. Result is: '0'.
2023-03-21 10:20:56,425 - INFO - '[User: testuser-16] [Task: func delete_all_projects]: finished. Result is: '0'.

2023-03-21 10:21:47,343 - INFO - '[User: testuser-16] [Task: func new_project 2]: finished. Result is: '0'.
2023-03-21 10:21:47,351 - INFO - '[User: testuser-15] [Task: func new_project 2]: finished. Result is: '0'.'.

2023-03-21 10:26:28,922 - INFO - '[User: testuser-16] [Task: func new_app 2]: finished. Result is: '0'.
2023-03-21 10:28:28,978 - INFO - '[User: testuser-15] [Task: func new_app 2]: finished. Result is: '0'.

The tasks run in loops Loop 1

2023-03-21 10:26:42,044 - INFO - '[User: testuser-16] [Task: func load_app 2 10]: finished. Result is: '0'.
2023-03-21 10:29:23,022 - INFO - '[User: testuser-16] [Task: func scale_deployment 2 2]: finished. Result is: '0'.
2023-03-21 10:30:33,654 - INFO - '[User: testuser-16] [Task: func scale_deployment 2 1]: finished. Result is: '0'.

2023-03-21 10:28:39,527 - INFO - '[User: testuser-15] [Task: func load_app 2 10]: finished. Result is: '0'.
2023-03-21 10:31:20,493 - INFO - '[User: testuser-15] [Task: func scale_deployment 2 2]: finished. Result is: '0'.
2023-03-21 10:32:31,110 - INFO - '[User: testuser-15] [Task: func scale_deployment 2 1]: finished. Result is: '0'.

Loop2

2023-03-21 10:31:16,133 - INFO - '[User: testuser-16] [Task: func load_app 2 10]: finished. Result is: '0'.
2023-03-21 10:33:57,054 - INFO - '[User: testuser-16] [Task: func scale_deployment 2 2]: finished. Result is: '0'.
2023-03-21 10:35:07,669 - INFO - '[User: testuser-16] [Task: func scale_deployment 2 1]: finished. Result is: '0'.

2023-03-21 10:32:51,388 - INFO - '[User: testuser-15] [Task: func load_app 2 10]: finished. Result is: '0'.
2023-03-21 10:35:32,311 - INFO - '[User: testuser-15] [Task: func scale_deployment 2 2]: finished. Result is: '0'.
2023-03-21 10:36:42,925 - INFO - '[User: testuser-15] [Task: func scale_deployment 2 1]: finished. Result is: '0'.

.... Post tasks run only once for each user, after halt is touched

2023-03-21 10:37:17,035 - INFO - '[User: testuser-15] [Task: func delete_project 2]: finished. Result is: '0'.
2023-03-21 10:39:54,506 - INFO - '[User: testuser-16] [Task: func delete_project 2]: finished. Result is: '0'.

Other group that does not include pre_tasks and post_tasks still work well

2023-03-21 10:21:37,070 - INFO - '[User: kubeadmin] [Task: func check_operators]: finished. Result is: '0'.
2023-03-21 10:31:47,461 - INFO - '[User: kubeadmin] [Task: func check_operators]: finished. Result is: '0'.

Test result

[Function]               |     Total|    Passed|    Failed|Failure Rate|
-----------------------------------------------------------------------
[delete_all_projects]    |         2|         2|         0|      0.0%|
[new_project]            |         4|         4|         0|      0.0%|
[check_operators]        |         2|         2|         0|      0.0%|
[new_app]                |         4|         4|         0|      0.0%|
[load_app]               |       110|       109|         1|      0.9%|
[scale_deployment]       |        20|        20|         0|      0.0%|
[delete_project]         |         4|         4|         0|      0.0%|
-----------------------------------------------------------------------
qiliRedHat commented 1 year ago

@svetsa-rh Hi Sharada, would you help to review this PR?

qiliRedHat commented 1 year ago

@paigerube14 PTAL when you have time.

qiliRedHat commented 1 year ago

Latest passed test thread https://redhat-internal.slack.com/archives/C0266JJ4XM5/p1679459581695479

paigerube14 commented 1 year ago

/lgtm