bcgov / nr-spar-data

Apache License 2.0
0 stars 0 forks source link

PoC - Data Synchronization using Seedlot table between Postgres and Oracle Databases in QA environment [Batch execution] #51

Closed marcionemec-daitan closed 2 months ago

marcionemec-daitan commented 4 months ago

After history load data from Seedlot tables in Oracle and Postgres Databases, data synchronization should be able to capture changes (updates/inserts) between both databases and spread the changes.

It will be done when:

1) All updated seedlot data in Oracle should change the right records in Postgres 2) All updated seedlot data in Postgres should change the right records in Oracle 3) All new seedlot data in Oracle should change the right records in Postgres 4) All new seedlot data in Postgres should change the right records in Oracle

marcionemec-daitan commented 2 months ago

Local tests found some discrepancies between Postgres and Oracle regarding Seedlot table described below:


--- Columns in SEEDLOT TABLE THAT EXISTS IN POSTGRES BUT NOT FOUND IN ORACLE: area_of_use_comment column interm_strg_locn_code non_orchard_pollen_contam_pct temporary_strg_client_number temporary_strg_locn_code


--- Columns in SEEDLOT TABLE THAT EXISTS IN ORACLE BUT NOT FOUND IN POSTGRES: --- POSTGRES SPAR SEEDLOT TABLE

COANCESTRY 
COLLECTION_AREA_RADIUS
COLLECTION_BGC_IND
COLLECTION_CLI_LOCN_CD
COLLECTION_SEED_PLAN_ZONE_IND
COLLECTION_SOURCE_CODE
COLLECTION_STANDARD_MET_IND
CONE_COLLECTION_METHOD2_CODE
CONE_COLLECTION_METHOD_CODE
CONE_SEED_DESC
EXTRACTION_COMMENT
EXTRACTION_VOLUME
FS721A_SIGNED_IND
HISTORICAL_TSR_DATE
INTERM_STRG_CLIENT_LOCN
INTERM_STRG_CMT
LNGTERM_STRG_ST_DATE
NAD_DATUM_CODE
NMBR_TREES_FROM_CODE
ORCHARD_COMMENT
ORCHARD_CONTAMINATION_PCT
ORCHARD_ID
ORG_UNIT_NO
ORIGINAL_SEED_QTY
PRICE_COMMENT
PRICE_PER_KG
PROVENANCE_ID
REGISTERED_DATE
REGISTERED_SEED_IND
SECONDARY_ORCHARD_ID
SEED_COAST_AREA_CODE
SEED_PLAN_ZONE_CODE
SEED_STORE_CLIENT_LOCN
SEED_STORE_CLIENT_NUMBER
STORED_CLI_LOCN_CD
STORED_CLI_NUMBER
SUPERIOR_PRVNC_IND
UTM_EASTING
UTM_NORTHING
UTM_ZONE

To continue tests, synchronization will be done only on matching columns between databases.

CC: @RMCampos , @craigyu