Prior to this change, we were doing case sensitive equality checks of non-key columns.
This makes the utility more restrictive, as Spark does not care about the casing of column names.
With this change, we rely on Spark to check if a column exists in the given dataframe. If Spark can find the column, we can proceed with the rest of the check.
Issue #, if available:
N/A
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Description of changes:
Issue #, if available: N/A
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.