Initializes a Spark session and defines the JDBC connection properties for source and target databases of CDM instance. The updates ensure that the correct output with some stat report, and environment variables are used for the JDBC connection properties. Additionally, the changes handle a scenario where a table does not exist in the source database by skipping to the next table.
Heartbeat function was introduce to ensure that the Spark job is running, by default the heartbeat would ticking every 30 seconds as example below:
Also implement CLI based approaches, to access this feature by python3 main.py transfer as CLI example from previous PR.
Initializes a Spark session and defines the JDBC connection properties for source and target databases of CDM instance. The updates ensure that the correct output with some stat report, and environment variables are used for the JDBC connection properties. Additionally, the changes handle a scenario where a table does not exist in the source database by skipping to the next table.
Heartbeat function was introduce to ensure that the Spark job is running, by default the heartbeat would ticking every 30 seconds as example below:
Also implement CLI based approaches, to access this feature by
python3 main.py transferas CLI example from previous PR.