[BUG]: Need help installing Recon

[ludovicc]

Is there an existing issue for this?

• [X] I have searched the existing issues

Category of Bug / Issue

ReconcileError

Current Behavior

Hello,

I'm trying to use Recon to validate our CDC pipeline which send data from MariaDB to Databricks with Unity. I have setup a foreign catalog to MariaDB since MySQL is not supported by Recon yet. I'm now trying to configure Recon to work with this setup, but if fails with an error message not really useful:

databricks -p prod labs remorph reconcile

Cannot find recon table configuration in existing reconcile installation. Please provide the configuration file Error: exit status 1

My config:

reconcile.yml

data_source: databricks
database_config:
  source_schema: sga
  target_catalog: prod_ca_regional
  target_schema: landing
metadata_config:
  catalog: prod_ca_regional
  schema: reconcile
  volume: reconcile_volume
report_type: all
secret_scope: p-cac-b05-dhubreg-kv-kvd
version: 1

recon_config_databricks_prod_ca_regional_all.json

{
  "source_catalog": "mariadb_ca",
  "source_schema": "sga",
  "target_catalog": "prod_ca_regional",
  "target_schema": "landing",
  "tables": [...]
}

Expected Behavior

An error message that provides the names of the expected configuration files.

Steps To Reproduce

No response

Relevant log output or Exception details

No response

Sample Query

No response

Operating System

macOS

Version

latest via Databricks CLI

[vijaypavann-db]

Hi @ludovicc, As mentioned here, recon_config filename should be of the following pattern: recon_config_<DATA_SOURCE>_<SOURCE_CATALOG_OR_SCHEMA>_<REPORT_TYPE>.json.

But the filename, that you have provided has <TARGET_CATALOG> name recon_config_databricks_prod_ca_regional_all.json. Please update it to recon_config_databricks_ mariadb_ca_all.json.

Please let us know, if you still face any issues after the update.

Will update the logging part.

Thanks

[ludovicc]

@vijaypavann-db I created a little patch for this. Thanks.