search

open-metadata / OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.
https://open-metadata.org
Apache License 2.0
5.59k stars 1.05k forks source link

Add Iceberg Profiler Support #17301

Open Prajwal214 opened 3 months ago

Prajwal214 commented 3 months ago

Affected module Does it impact the UI, backend or Ingestion Framework? UI

Describe the bug Currently, Profiler and Data Quality features are not supported for Iceberg connectors. However, the option to create test cases is still visible in the UI.

To Reproduce

image

Expected behavior Is it possible to hide the Profiler & Data Quality tab for Iceberg connectors since these features are not yet supported?

Version:

Additional context Add any other context about the problem here.

nqvuong1998 commented 2 months ago

Hi @TeddyCr , Iceberg stores schema and column statistics in manifest files. OpenMetadata can leverage this information for metadata ingestion and data profiling.