search

GoogleCloudPlatform / datacatalog-connectors-bi

Sample code with integration between Data Catalog and BI data sources.
Apache License 2.0
32 stars 16 forks source link

Add the list-elasticube-deps CLI option #102

Closed ricardolsmendes closed 3 years ago

ricardolsmendes commented 3 years ago

- What I did

  1. Enhanced Sisense2DataCatalog's CLI by adding the list-elasticube-deps option.
  2. Improved the ElastiCube dependency finding results output by tabulating the Matching field or filter, Table, and Column Tag fields.

- How I did it

  1. Improved sisense.Sisense2DataCatalogCli arg parsers helped by the new _ListElastiCubeDepsCliHelper class, responsible for setting up the CLI tool introduced in the present PR.
  2. Added the tabulate library to improve addons.ElastiCubeDependencyPrinter's results output.
  3. Added unit tests that fully cover the above code changes.
  4. Updated README.md by adding instructions on how to use the CLI.

- How to verify it Run the unit tests and, if possible, the connector in an integrated environment. For the new list-elasticube-deps option, the output should be similar to:

INFO:root:
INFO:root:The entries.lookup API does not work for user-specified Entries
 (https://issuetracker.google.com/issues/202510978 for details). Catalog search is used as a fallback.
INFO:root:
INFO:root:===> Searching Data Catalog...
INFO:root:Query: system=Sisense (type=Dashboard OR type=Widget) tag:id:__REDACTED__
INFO:root:==== DONE ========================================
INFO:root:     8 results found
INFO:root:
INFO:root:===> Getting entries...
INFO:root:==== DONE ========================================
INFO:root:
INFO:root:===> Listing tags...
INFO:root:==== DONE ========================================
INFO:root:
INFO:root:===> Assembling results...
INFO:root:==== DONE ========================================

Results:

1
Entry       : projects/__REDACTED__/locations/us/entryGroups/sisense/entries/sss___REDACTED__
Type        : Dashboard
Title       : Sample - Healthcare
Data source : Sample Healthcare
URL         : https://__REDACTED__.sisense.com/app/main#/dashboards/__REDACTED__

 Matching field or filter                   | Table      | Column
--------------------------------------------+------------+----------------
 filters.gender                             | Patients   | Gender
 filters.division                           | Divisions  | Divison_name
 filters.admission date                     | Admissions | Admission_Time
 filters.top diagnoses                      | Diagnosis  | Description

...

- Description for the changelog Enhanced Sisense2DataCatalog's CLI by adding the list-elasticube-deps option.

This PR is part of the effort to deliver #70.