CityofPittsburgh / data-rivers

Apache Airflow and Beam ETL scripts for the City of Pittsburgh's data analysis pipelines
10 stars 1 forks source link

Allow Specification of Conditional Fields in Data Quality Query Builder #707

Closed jasonfic closed 7 months ago

jasonfic commented 7 months ago

Previously, the Data Quality query function required you to check if every selected field was null when performing a data quality check (for example, if you wanted to inspect changes to ID, Name, and Job Title, the query would have to check whether all 3 columns were null). This updated function allows you to select 5 fields but only check if 3 of them are null if it is important to display a longer list of columns in the output data quality table