Good day - I am evaluating DataCleaner as a potential means of identifying data quality issues in source data, prior to a data migration. I have added various filters, each of which is checking a certain aspect of the input data. For each record that 'fails' a filter (and, therefore, violates a Data Quality rule), I am writing the record to a staging table. There are several such filters in my job, each of which tests a different rule, but each violated record gets written to the same staging table. Is there any relatively easy way to add a field to the output from each filter that defines which filter (i.e. which rule) the record failed on?
Good day - I am evaluating DataCleaner as a potential means of identifying data quality issues in source data, prior to a data migration. I have added various filters, each of which is checking a certain aspect of the input data. For each record that 'fails' a filter (and, therefore, violates a Data Quality rule), I am writing the record to a staging table. There are several such filters in my job, each of which tests a different rule, but each violated record gets written to the same staging table. Is there any relatively easy way to add a field to the output from each filter that defines which filter (i.e. which rule) the record failed on?