Some CI tests are unreliable. Current situation, from @shoriminimoe
I looked through the last 5 automated runs of the integration tests. Although there are a few flaky tests, I haven't found any showstoppers. For reference, these are the tests that failed along with their failure rate.
Results are listed as "name: rate" e.g. "test: 1/5".
- integ-run (<https://10.5.11.200/>) / ansible-test (vm_disk\_\_remove_disk): 1/5
- integ-run (<https://10.5.11.201/>) / ansible-test (vm_disk\_\_remove_disk): 1/5
- integ-run (<https://10.5.11.203/>) / ansible-test (iso): 1/5
- integ-run (<https://10.5.11.203/>) / ansible-test (virtual_disk): 1/5
- integ-seq-run (<https://10.5.11.200/>) / ansible-test (oidc_config): 2/5
- integ-seq-run (<https://10.5.11.201/>) / ansible-test (oidc_config): 1/5
- integ-seq-run (<https://10.5.11.201/>) / ansible-test (role_cluster_config): 1/5
- examples-run (2.16.0, syslog_server.yml): 2/5
- examples-run (2.16.0, support_tunnel.yml): 4/5
- examples-run (2.16.0, shutdown_restart_tagged_vms.yml): 3/5
- examples-run (2.16.0, vm_bios_to_uefi.yml): 5/5
- examples-run (2.16.0, vm_nic.yml): 5/5
- examples-run (2.16.0, vm.yml): 5/5
CI test improvements
Some CI tests are unreliable. Current situation, from @shoriminimoe
TODO: