Document the list of technologies that broke during Openshift upgrades

[mitovskaol]

In preparation to hiring an Automation Tester, we need to have a list of technologies and products/tools that are currently implemented on the platform that have a historic record of breaking during Openshift upgrade. The automation tester will develop a test framework and a few test cases that will test the availability of a particular tech/tool/service after an Openshift upgrade is applied in KLAB cluster.

Additional context In addition to the technology name, we need to capture the impact that the tech outage has on the platform and how many apps were impacted. This information will help prioritize the development of test cases starting with with those that test the availability of the tech that may have the broadest impact during its service disruption. i.e. When Patroni broke, multiple teams were impacted, so it would be a good idea to choose Patroni as one of the first test cases.

Most information should be available from the post-mortem docs and ZenHub tickets.

DoD:

• [ ] The following info is captured in this ticket: Name of tech/product/tool that had an outage after an Openshift upgrade, impact level (low - 1-2 teams was impacted, medium - multiple teams were impacted, high - most of the product teams were impacted), outage occurrence (once, multiple times)

[StevenBarre]

Vault - low - once

I think I remember an issue where after upgrade the vault injector was not working and needed some tweaks, but was not detected during LAB because we didn't have a good check.

[StevenBarre]

Porter - low - once

One time after the upgrade the porter operator seems to have deleted and recreated objects, causing port numbers for some services to change.

[mitovskaol]

Patroni - high - twice(?)

[StevenBarre]

Patroni and porter again

[StevenBarre]

Automation Tester was hired and created an N8N based test tool