K8SPG-600: Fix failed backup status

https://perconadev.atlassian.net/browse/K8SPG-600

DESCRIPTION

Problem: If the user has the wrong pgbackrest credentials, the failed backup doesn't get the failed state in the pg-backup .status..

Reason: When incorrect pgbackrest credentials are used, the pg cluster reconcile fails, which stops the process of updating crunchy's postgrescluster resource. It is necessary to delete backup annotations from crunchy's resource, and this problem doesn't allow it to continue.

Solution: Don't stop the reconcile process with the pgbackrest error. Log the error message instead. Also, the operator should avoid trying to use pgbackrest when the backup job has failed.

CHECKLIST

Jira

[x] Is the Jira ticket created and referenced properly?
[x] Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
[x] Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

[ ] Is an E2E test/test case added for the new feature/change?
[ ] Are unit tests added where appropriate?

Config/Logging/Testability

[x] Are all needed new/changed options added to default YAML files?
[x] Did we add proper logging messages for operator actions?
[x] Did we ensure compatibility with the previous version or cluster upgrade process?
[x] Does the change support oldest and newest supported PG version?
[x] Does the change support oldest and newest supported Kubernetes version?

Test name	Status
custom-extensions	passed
demand-backup	passed
init-deploy	passed
major-upgrade	passed
monitoring	passed
one-pod	passed
operator-self-healing	passed
pitr	passed
scaling	passed
scheduled-backup	passed
self-healing	passed
start-from-backup	passed
tablespaces	passed
telemetry-transfer	passed
upgrade-consistency	passed
upgrade-minor	passed
users	passed
We run 17 out of 17

percona / percona-postgresql-operator

K8SPG-600: Fix failed backup status #820

DESCRIPTION

CHECKLIST