zalando / postgres-operator

Postgres operator creates and manages PostgreSQL clusters running in Kubernetes
https://postgres-operator.readthedocs.io/
MIT License
4.37k stars 980 forks source link

fix sync streams and add diffs for annotations and owner references #2728

Closed FxKu closed 3 months ago

FxKu commented 3 months ago

in our internal e2e tests I found the following logs:

level=warning msg="database replication slots for streams with applicationId test-app not in sync, skipping event stream sync" 
level=warning msg="database replication slots for streams with applicationId test-app-2 not in sync, skipping event stream sync"

The stream with test-app-2 uses a slot with not existing schema and table hence the expected second log line. But the stream of the first app should be fine. I have extended and improved the TestHasSlotsInSync unit test but found no bugs. After adding a logger, I found the issue:

In #2704 we removed always setting a slot to sync in order not block syncing broken streams. However, now they are only set on create, update or delete, but not on a normal sync. Therefore, hasSlotsInSync does not find the desired slots in slotsToSync and skips the stream sync. The proposed solution sets a slot if

If we find out later that the publication can be removed it is set to nil. This code already exists, too.

I'm using this PR to also introduce diffs for annotations and owner references in streams. It required some rewriting of unit tests.

The changes on maintenanceWindows function were triggered because the fake stream client was used before which wasn't working anymore after my changes. First, I choose another client from the same go files (newFakeK8sAnnotationsClient) but then I realized, it's better to just pass the maintenance windows from the manifest to this function to simplify the code.

idanovinda commented 3 months ago

👍

FxKu commented 3 months ago

:+1: