Allow Column Level profiler configuration at Profiler workflow

open-metadata / OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.

https://open-metadata.org

Apache License 2.0

5.39k stars 1.02k forks source link

Allow Column Level profiler configuration at Profiler workflow #16550

Open TeddyCr opened 4 months ago

TeddyCr commented 4 months ago

Context As a user i want to be able to exclude column computation using actual column name or column name pattern in the profiler configuration

A.C.

Setting logic to exclude columns
Ingestion logic to apply column filter exclusion
UI component

harshach commented 2 months ago

capturing more requirements

Simple flag to remove column level metric computation all together
Allow users to exclude/include column level metrics

Any changes at profiler workflow level will overrride the changes from individual table/database/schema level configs

TeddyCr commented 2 months ago

Allow users to exclude/include column level metrics

Are you thinking of an include/exclude patterns ?