search

GoogleCloudPlatform / datacatalog-connectors-bi

Sample code with integration between Data Catalog and BI data sources.
Apache License 2.0
32 stars 16 forks source link

[BUG] Initial full sync entries in data catalog are not searchable #11

Closed nagendra7db closed 4 years ago

nagendra7db commented 4 years ago

After the successful initial/first run of tableau-connector entries like dashboards ingested into google datacatalog, searching by entries or by tags will not result. But the entries exist under the entry_group=tableau.

Expected: Search in datacatalog by entry name (like dashboard name) or by tag-template "tableau_dashboard_metadata" should return list of entries/views/dashboards/tables...etc

To Reproduce: Create a Datacatalog service account with catalog admin role : roles/datacatalog.admin Create a developer tableau demo with example workbook. Execute exact steps as in readme document to setup env, packages and connector execution

Couple of things to note:

  1. After the first run and entries are created in catalog a. either add a new dashboard to existing workbook in tableau and rerun the connector. Now the all existing dashboards and new dashboard under the same workbook are searchable in datacatalog b. After initial/first run, when entries are not searchable, edit tag-template value for one entry in datacatalog and save. Now that entry is searchable

  2. I have noticed this pattern of initial/first entries not searchable even with other connectors like datacatalog-connectors-rdbms/google-datacatalog-mysql-connector where mysql database or tables or columns or tag-templates are not searchable unless there is a change in DB or meta of that entry in catalog. Which make me assume its either an issue in google datacatalog or in ingestion code: datacatalog-connectors/google-datacatalog-connectors-commons/

github-actions[bot] commented 4 years ago

Thanks for submiting an issue for the first time! We will take a look at it.

mesmacosta commented 4 years ago

Thanks for all the info @nagendra7db, we are analyzing it.

Did you notice if you don't do any changes like:

b. After initial/first run, when entries are not searchable, edit tag-template value for one entry in datacatalog and save. Now that entry is searchable

it shows up after a while?

nagendra7db commented 4 years ago

Yes, I waited as long as 15hours and still they are not searchable. Once either of those changes made from tableau end add new dashboard or edit existing entry, they are immediately searchable.

Thanks

mesmacosta commented 4 years ago

@nagendra7db The issue was fixed, I will close this. But please run the connector again, and if you are still experiencing the issue reopen it.

Thanks for your help and for filling this.