This pull request addresses a bug in the MergeIntoNodeBuilder, where the functionalExpressions were overridden with an empty sequence.
Based on the merge semantic, it is correct to have an arbitrary expression in the UPDATE clause. For instance, let's consider a scenario where we have a source table with the schema id long, name string and a target delta table with schema id long, text string. In this case, a valid merge operation could be:
MERGE INTO target
USING source
ON target.id = source.id
WHEN MATCHED THEN UPDATE SET text = lower(source.name)
If functionalExpressions is empty, the output attribute text will have a child expression id resulting from the conversion of lower(source.name), which is not referenced in the expression.functions output lineage section.
This pull request addresses a bug in the
MergeIntoNodeBuilder
, where the functionalExpressions were overridden with an empty sequence.Based on the merge semantic, it is correct to have an arbitrary expression in the UPDATE clause. For instance, let's consider a scenario where we have a source table with the schema
id long, name string
and a target delta table with schemaid long, text string
. In this case, a valid merge operation could be:If functionalExpressions is empty, the output attribute
text
will have a child expression id resulting from the conversion oflower(source.name)
, which is not referenced in theexpression.functions
output lineage section.