GoogleCloudPlatform / datacatalog-connectors-bi

Sample code with integration between Data Catalog and BI data sources.
Apache License 2.0
32 stars 16 forks source link

Add the find-elasticube-deps CLI option #99

Closed ricardolsmendes closed 3 years ago

ricardolsmendes commented 3 years ago

- What I did Enhanced Sisense2DataCatalog's CLI by adding the find-elasticube-deps option. The original catalog synchronization CLI tool is now under the sync-catalog option.

- How I did it

  1. Refactored and improved sisense.Sisense2DataCatalogCli arg parsers. Two helper classes were created, _SyncCatalogCliHelper and _FindElastiCubeDepsCliHelper, each one setting up a specific CLI tool.
  2. Added addons.ElastiCubeDependencyPrinter in order to print dependency finding results in a user-friendly fashion.
  3. Added unit tests that fully cover the above code changes.
  4. Update README.md by adding instructions on how to use the CLI.

- How to verify it Run the unit tests and, if possible, the connector in an integrated environment. For the new find-elasticube-deps option, the output should be similar to:

INFO:root:
INFO:root:===> Searching Data Catalog...
INFO:root:Query: system=Sisense (type=Dashboard OR type=Widget)  tag:table:"admissions" tag:column:"Admission_Time"
INFO:root:==== DONE ========================================
INFO:root:
INFO:root:Catalog search returned 10 results
INFO:root:
INFO:root:===> Getting entries...
INFO:root:==== DONE ========================================
INFO:root:
INFO:root:===> Listing tags...
INFO:root:==== DONE ========================================
INFO:root:
INFO:root:===> Assembling results...
INFO:root:==== DONE ========================================

Results:

1
Entry       : projects/__REDACTED__/locations/us/entryGroups/sisense/entries/sss__REDACTED__
Type        : Widget
Title       : TOP 10 DIAGNOSIS
Dashboard   : Sample - Medical
Data source : Sample Healthcare
URL         : https://__REDACTED__.sisense.com/app/main#/dashboards/__REDACTED__/widgets/__REDACTED__

Matching fields and filters:

  Component : fields.avg days admitted.formula.days in admission  time
  Table     : Admissions
  Column    : Admission_Time (Calendar)
--------------------------------------------------
...

- Description for the changelog Enhanced Sisense2DataCatalog's CLI by adding the find-elasticube-deps option.

This PR is part of the effort to deliver #70.