K8SPXC-1357: don't restart haproxy on pass change

[pooknull]

https://perconadev.atlassian.net/browse/K8SPXC-1357

DESCRIPTION

Problem: When the monitoring user's password for monitoring is changed, the HAProxy pods are restarted. This is unnecessary because HAProxy reads the password from the file /etc/mysql/mysql-users-secret/monitor every time it performs an operation.

Solution: Do not add the last-applied-secret annotation to the HAProxy statefulset template when the password is changed: https://github.com/percona/percona-xtradb-cluster-operator/pull/1743/commits/4a5e19984efd817c80eb613db75678287db003fd

But we should add this annotation if MySQL 5.7 is used: https://github.com/percona/percona-xtradb-cluster-operator/pull/1743/commits/ccd3970c49ddd8727c808bb2939ddc7c245a00d9

If the pmm secret is changed, HAProxy should still be restarted on any version.

Additionally, this PR addresses an issue in the updatePod function where it often failed due to conflict errors. This problem caused frequent failures in monitoring test, because some statefulsets missed necessary annotations due to this error. To resolve this, the operator should use the RetryOnConflict function: https://github.com/percona/percona-xtradb-cluster-operator/pull/1743/commits/a49314047286701973d8dce95162cf0ac4a7eaa5

The updatePod function's code became more complex with this change and had a lot of duplicated code with the pxc.StatefulSet function. To improve it, the updatePod function should use the pxc.StatefulSet function instead of duplicated code: https://github.com/percona/percona-xtradb-cluster-operator/pull/1743/commits/b8fd22a10b4b33ebd1993a6c022dba2bbfbfe517

CHECKLIST

Jira

• [x] Is the Jira ticket created and referenced properly?
• [x] Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
• [x] Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

• [x] Is an E2E test/test case added for the new feature/change?
• [ ] Are unit tests added where appropriate?
• [x] Are OpenShift compare files changed for E2E tests (compare/*-oc.yml)?

Config/Logging/Testability

• [x] Are all needed new/changed options added to default YAML files?
• [x] Did we add proper logging messages for operator actions?
• [x] Did we ensure compatibility with the previous version or cluster upgrade process?
• [x] Does the change support oldest and newest supported PXC version?
• [x] Does the change support oldest and newest supported Kubernetes version?

[JNKPercona]

Test name	Status
affinity-8-0	passed
auto-tuning-8-0	passed
cross-site-8-0	passed
demand-backup-cloud-8-0	passed
demand-backup-encrypted-with-tls-8-0	passed
demand-backup-8-0	passed
haproxy-5-7	passed
haproxy-8-0	passed
init-deploy-5-7	passed
init-deploy-8-0	passed
limits-8-0	passed
monitoring-2-0-8-0	passed
one-pod-5-7	passed
one-pod-8-0	passed
pitr-8-0	passed
pitr-gap-errors-8-0	passed
proxy-protocol-8-0	passed
proxysql-sidecar-res-limits-8-0	passed
pvc-resize-5-7	passed
pvc-resize-8-0	passed
recreate-8-0	passed
restore-to-encrypted-cluster-8-0	passed
scaling-proxysql-8-0	passed
scaling-8-0	passed
scheduled-backup-5-7	passed
scheduled-backup-8-0	passed
security-context-8-0	passed
smart-update1-8-0	passed
smart-update2-8-0	passed
storage-8-0	passed
tls-issue-cert-manager-ref-8-0	passed
tls-issue-cert-manager-8-0	passed
tls-issue-self-8-0	passed
upgrade-consistency-8-0	passed
upgrade-haproxy-5-7	passed
upgrade-haproxy-8-0	passed
upgrade-proxysql-5-7	passed
upgrade-proxysql-8-0	passed
users-5-7	passed
users-8-0	passed
validation-hook-8-0	passed
We run 41 out of 41

commit: https://github.com/percona/percona-xtradb-cluster-operator/pull/1743/commits/51af051703ea68e6367821ec3068826842ab5c9d image: perconalab/percona-xtradb-cluster-operator:PR-1743-51af0517