in our internal e2e tests I found the following logs:
level=warning msg="database replication slots for streams with applicationId test-app not in sync, skipping event stream sync"
level=warning msg="database replication slots for streams with applicationId test-app-2 not in sync, skipping event stream sync"
The stream with test-app-2 uses a slot with not existing schema and table hence the expected second log line. But the stream of the first app should be fine. I have extended and improved the TestHasSlotsInSync unit test but found no bugs. After adding a logger, I found the issue:
In #2704 we removed always setting a slot to sync in order not block syncing broken streams. However, now they are only set on create, update or delete, but not on a normal sync. Therefore, hasSlotsInSync does not find the desired slots in slotsToSync and skips the stream sync. The proposed solution sets a slot if
create and alter operations succeed (behavior not changed)
or if current publication is not new or tables changes (new, now with else, prior to #2704 it was done for every publication).
If we find out later that the publication can be removed it is set to nil. This code already exists, too.
I'm using this PR to also introduce diffs for annotations and owner references in streams. It required some rewriting of unit tests.
The changes on maintenanceWindows function were triggered because the fake stream client was used before which wasn't working anymore after my changes. First, I choose another client from the same go files (newFakeK8sAnnotationsClient) but then I realized, it's better to just pass the maintenance windows from the manifest to this function to simplify the code.
in our internal e2e tests I found the following logs:
The stream with test-app-2 uses a slot with not existing schema and table hence the expected second log line. But the stream of the first app should be fine. I have extended and improved the TestHasSlotsInSync unit test but found no bugs. After adding a logger, I found the issue:
In #2704 we removed always setting a slot to sync in order not block syncing broken streams. However, now they are only set on create, update or delete, but not on a normal sync. Therefore, hasSlotsInSync does not find the desired slots in slotsToSync and skips the stream sync. The proposed solution sets a slot if
If we find out later that the publication can be removed it is set to nil. This code already exists, too.
I'm using this PR to also introduce diffs for annotations and owner references in streams. It required some rewriting of unit tests.
The changes on maintenanceWindows function were triggered because the fake stream client was used before which wasn't working anymore after my changes. First, I choose another client from the same go files (newFakeK8sAnnotationsClient) but then I realized, it's better to just pass the maintenance windows from the manifest to this function to simplify the code.