In the current codebase, the reconcile_data and _get_mismatch_data functions are tightly integrated with HASH_COLUMN_NAME to identify the mismatch, missing_in_src and missing_in_tgt records; due to this, it can't be reused for any other use cases like Aggregate Data Reconciliation .
Proposed Solution
Need to update these as generic common functions, so that it can be reused for other use cases like Aggregate Data Reconciliation along with Hash Data Reconciliation.
Additional Context
For Aggregate Data Reconciliation, two new functions, join_aggregate_data and _get_mismatch_agg_data, are introduced. Once generic functions are developed, these two functions should be removed and integrated with the common functions.
Is there an existing issue for this?
Category of feature request
Reconcile
Problem statement
In the current codebase, the
reconcile_data
and_get_mismatch_data
functions are tightly integrated withHASH_COLUMN_NAME
to identify themismatch
,missing_in_src
andmissing_in_tgt
records; due to this, it can't be reused for any other use cases likeAggregate Data Reconciliation
.Proposed Solution
Need to update these as generic common functions, so that it can be reused for other use cases like
Aggregate Data Reconciliation
along withHash Data Reconciliation
.Additional Context
For
Aggregate Data Reconciliation,
two new functions,join_aggregate_data
and_get_mismatch_agg_data
, are introduced. Once generic functions are developed, these two functions should be removed and integrated with the common functions.