open-metadata / OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.
https://open-metadata.org
Apache License 2.0
5.3k stars 998 forks source link

[Data Quality] Column is displayed for unsupported columns #16699

Closed sushi30 closed 2 months ago

sushi30 commented 3 months ago

Affected module UI

Describe the bug When defining a column test, I expect only relevant tests to be offered given the column data type. For example, the columnValuesToMatchRegex is only expected to run on STRING types yet I am offered to run it on a numeric type in this example: image

Expected behavior For a column test, display only the tests that apply to the column data type.

Version:

TeddyCr commented 3 months ago
{
  "name": "columnValuesToMatchRegex",
  "fullyQualifiedName": "columnValuesToMatchRegex",
  "displayName": "Column Values To Match Regex Pattern",
  "description": "This schema defines the test ColumnValuesToMatchRegex. Test the values in a column to match a given regular expression. ",
  "entityType": "COLUMN",
  "testPlatforms": ["OpenMetadata"],
  "supportedDataTypes": ["BYTES", "STRING", "MEDIUMTEXT", "TEXT", "CHAR", "VARCHAR"],
  "parameterDefinition": [
    {
      "name": "regex",
      "displayName": "RegEx Pattern",
      "description": "The regular expression the column entries should match. For database without regex support (i.e. MSSQL, AzureSQL) this test will use `LIKE`.",
      "dataType": "STRING",
      "required": true
    }
  ],
  "supportsRowLevelPassedFailed": true,
  "provider": "system"
}

The test definition has the supported dtype, we should check if it is enforced correctly from the ui