When a user adds a sourceCapture we will then need to show them a setting to control the deltaUpdates flag. This is setting ON the sourceCapture setting itself is really controlling if fields added has the binding deltaUpdates defaulted.
When adding a manual binding we will need to get the deltaUpdate and targetSchema settings for the bindings.
We will only default new bindings when selecting a Source Capture
deltaUpdates: true, then new bindings that are added automatically by sourceCapture, are added with delta updates set to true.
targetSchema: "fromSource", then new bindings that are added automatically by sourceCapture, are added with their schema set to the 2nd-to-last part of the collection name
(the alternative value of targetSchema is leaveEmpty which leaves the schema empty for that binding, thus defaulting to the global schema of the materialization task (current behavior)
Previous Ask
We want to start defaulting delta_updates to true for materializations. While doing this work we also want to expose this setting in the form (somewhere) for the user to easily edit.
travjenkins
commented
1 week ago
Current Ask
sourceCapturewe will then need to show them a setting to control thedeltaUpdatesflag. This is setting ON thesourceCapturesetting itself is really controlling if fields added has the bindingdeltaUpdatesdefaulted.deltaUpdateandtargetSchemasettings for the bindings.We will only default new bindings when selecting a
Source CapturePrevious Ask
We want to start defaulting
delta_updatesto true for materializations. While doing this work we also want to expose this setting in the form (somewhere) for the user to easily edit.Docs: https://docs.estuary.dev/reference/Connectors/materialization-connectors/BigQuery/#delta-updates