DataHub validating booleans against wrong schema

[zaneselvans]

When uploading a tabular data resource containing boolean fields that have trueValues and falseValues set but that do not include the values True and False respectively, DataHub view generation fails, with errors of the variety:

ERROR :Failed to cast row: Field "FIELD_NAME_HERE" can't cast value "True" for type "boolean" with format "default"

It appears that the boolean values in the tabular data package have been converted internally during processing to True and False but that those values are being validated against or cast on the basis of the trueValues and falseValues attributes which are associated with the tabular data resource (which may not include True or False).

How to reproduce

• Create a tabular data package with a tabular data resource that contains boolean fields, specifying trueValues and falseValues attributes which are consistent with the data file contents, but which do not contain the common True and False strings.
• Call data push on the tabular data package.

Expected behavior

Validation and view generation should proceed normally, if the tabular data resource has appropriately defined trueValues and falseValues as they pertain to the packaged data, with casting and validation taking place on the basis of the valid values provided.

[zaneselvans]

It is possible to work around this issue by including True and False in the trueValues and falseValues arrays associated with the boolean fields, but this may mean introducing invalid metadata -- if True and False are not in fact valid within the packaged data. (e.g. if the data uses Y and N instead).

[rufuspollock]

@akariv do you have a sense of what is going on here?